org.tribuo.util.tokens.options

Class CoreTokenizerOptions

java.lang.Object

org.tribuo.util.tokens.options.CoreTokenizerOptions

All Implemented Interfaces:

com.oracle.labs.mlrg.olcut.config.Options, TokenizerOptions

public class CoreTokenizerOptions
extends Object
implements TokenizerOptions

CLI Options for all the tokenizers in the core package.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CoreTokenizerOptions.CoreTokenizerType Tokenizer type.

Field Summary

Fields

Modifier and Type	Field and Description
BreakIteratorTokenizerOptions	breakIteratorOptions
CoreTokenizerOptions.CoreTokenizerType	coreTokenizerType
SplitCharactersTokenizerOptions	splitCharactersTokenizerOptions
SplitPatternTokenizerOptions	splitPatternTokenizerOptions

Fields inherited from interface com.oracle.labs.mlrg.olcut.config.Options

header

Constructor Summary

Constructors

Constructor and Description
CoreTokenizerOptions()

Method Summary

Modifier and Type	Method and Description
Tokenizer	getTokenizer() Creates the appropriately configured tokenizer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Options

formatUsage, generateTypeDescription, getAllOptions, getEnumConstantString, getOptionFields, getOptions, getOptionsDescription, getOptionUsage, getOptionUsage, getOptionUsage, getUsage

Field Detail

breakIteratorOptions

public BreakIteratorTokenizerOptions breakIteratorOptions

splitCharactersTokenizerOptions

public SplitCharactersTokenizerOptions splitCharactersTokenizerOptions

splitPatternTokenizerOptions

public SplitPatternTokenizerOptions splitPatternTokenizerOptions

coreTokenizerType

@Option(longName="core-tokenizer-type",
        usage="Type of tokenizer")
public CoreTokenizerOptions.CoreTokenizerType coreTokenizerType

Constructor Detail

CoreTokenizerOptions

public CoreTokenizerOptions()

Method Detail

getTokenizer

public Tokenizer getTokenizer()

Description copied from interface: TokenizerOptions

Creates the appropriately configured tokenizer.

Specified by:

getTokenizer in interface TokenizerOptions

Returns:

The configured tokenizer.