org.tribuo.util.tokens.options

Class SplitCharactersTokenizerOptions

java.lang.Object

org.tribuo.util.tokens.options.SplitCharactersTokenizerOptions

All Implemented Interfaces:

com.oracle.labs.mlrg.olcut.config.Options, TokenizerOptions

public class SplitCharactersTokenizerOptions
extends Object
implements TokenizerOptions

CLI options for a SplitCharactersTokenizer.

Field Summary

Fields

Modifier and Type	Field and Description
char[]	splitChars
char[]	splitXDigitsChars

Fields inherited from interface com.oracle.labs.mlrg.olcut.config.Options

header

Constructor Summary

Constructors

Constructor and Description
SplitCharactersTokenizerOptions()

Method Summary

Modifier and Type	Method and Description
Tokenizer	getTokenizer() Creates the appropriately configured tokenizer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Options

formatUsage, generateTypeDescription, getAllOptions, getOptionFields, getOptions, getOptionsDescription, getOptionUsage, getOptionUsage, getOptionUsage, getUsage

Field Detail

splitChars

@Option(longName="sc-tokenizer-split-characters",
        usage="The characters to split on.")
public char[] splitChars

splitXDigitsChars

@Option(longName="sc-tokenizer-split-x-digits",
        usage="Characters to split on unless they appear between digits")
public char[] splitXDigitsChars

Constructor Detail

SplitCharactersTokenizerOptions

public SplitCharactersTokenizerOptions()

Method Detail

getTokenizer

public Tokenizer getTokenizer()

Description copied from interface: TokenizerOptions

Creates the appropriately configured tokenizer.

Specified by:

getTokenizer in interface TokenizerOptions

Returns:

The configured tokenizer.