Package org.tribuo.util.tokens.options

package org.tribuo.util.tokens.options

OLCUT Options implementations which can construct Tokenizers of various types.

Related Packages

Package	Description
org.tribuo.util.tokens	Core definitions for tokenization.
org.tribuo.util.tokens.impl	Simple fixed rule tokenizers.
org.tribuo.util.tokens.universal

Class	Description
BreakIteratorTokenizerOptions	CLI options for a BreakIteratorTokenizer.
CoreTokenizerOptions	CLI Options for all the tokenizers in the core package.
CoreTokenizerOptions.CoreTokenizerType
SplitCharactersTokenizerOptions	CLI options for a SplitCharactersTokenizer.
SplitPatternTokenizerOptions	CLI options for a SplitPatternTokenizer.
TokenizerOptions	CLI Options for creating a tokenizer.