org.tribuo.util.tokens.options

Interface TokenizerOptions

All Superinterfaces:

com.oracle.labs.mlrg.olcut.config.Options

All Known Implementing Classes:

BreakIteratorTokenizerOptions, CoreTokenizerOptions, SplitCharactersTokenizerOptions, SplitPatternTokenizerOptions

public interface TokenizerOptions
extends com.oracle.labs.mlrg.olcut.config.Options

CLI Options for creating a tokenizer.

Field Summary

Fields inherited from interface com.oracle.labs.mlrg.olcut.config.Options

header

Method Summary

Modifier and Type	Method and Description
Tokenizer	getTokenizer() Creates the appropriately configured tokenizer.

Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Options

formatUsage, generateTypeDescription, getAllOptions, getEnumConstantString, getOptionFields, getOptions, getOptionsDescription, getOptionUsage, getOptionUsage, getOptionUsage, getUsage

Method Detail

getTokenizer

Tokenizer getTokenizer()

Creates the appropriately configured tokenizer.

Returns:

The configured tokenizer.