Class BreakIteratorTokenizerOptions

java.lang.Object

org.tribuo.util.tokens.options.BreakIteratorTokenizerOptions

All Implemented Interfaces:

com.oracle.labs.mlrg.olcut.config.Options, TokenizerOptions

public class BreakIteratorTokenizerOptions
extends Object
implements TokenizerOptions

CLI options for a BreakIteratorTokenizer.

Field Summary

Fields

Modifier and Type	Field	Description
String	languageTag

Fields inherited from interface com.oracle.labs.mlrg.olcut.config.Options

header

Constructor Summary

Constructors

Constructor	Description
BreakIteratorTokenizerOptions()

Method Summary

Modifier and Type	Method	Description
Tokenizer	getTokenizer()	Creates the appropriately configured tokenizer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Options

getOptionsDescription

Field Details

languageTag

@Option(longName="bi-tokenizer-language-tag",
        usage="BreakIteratorTokenizer - The language tag of the locale to be used.")
public String languageTag

Constructor Details

BreakIteratorTokenizerOptions

public BreakIteratorTokenizerOptions()

Method Details

getTokenizer

public Tokenizer getTokenizer()

Description copied from interface: TokenizerOptions

Creates the appropriately configured tokenizer.

Specified by:

getTokenizer in interface TokenizerOptions

Returns:

The configured tokenizer.