Class CoreTokenizerOptions
java.lang.Object
org.tribuo.util.tokens.options.CoreTokenizerOptions
- All Implemented Interfaces:
com.oracle.labs.mlrg.olcut.config.Options
,TokenizerOptions
CLI Options for all the tokenizers in the core package.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic enum
Tokenizer type. -
Field Summary
FieldsModifier and TypeFieldDescriptionFields inherited from interface com.oracle.labs.mlrg.olcut.config.Options
header
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionCreates the appropriately configured tokenizer.Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Options
getOptionsDescription
-
Field Details
-
breakIteratorOptions
-
splitCharactersTokenizerOptions
-
splitPatternTokenizerOptions
-
coreTokenizerType
@Option(longName="core-tokenizer-type", usage="Type of tokenizer") public CoreTokenizerOptions.CoreTokenizerType coreTokenizerType
-
-
Constructor Details
-
CoreTokenizerOptions
public CoreTokenizerOptions()
-
-
Method Details
-
getTokenizer
Description copied from interface:TokenizerOptions
Creates the appropriately configured tokenizer.- Specified by:
getTokenizer
in interfaceTokenizerOptions
- Returns:
- The configured tokenizer.
-