Interface TokenizerOptions

All Superinterfaces:
com.oracle.labs.mlrg.olcut.config.Options
All Known Implementing Classes:
BreakIteratorTokenizerOptions, CoreTokenizerOptions, SplitCharactersTokenizerOptions, SplitPatternTokenizerOptions
public interface TokenizerOptions extends com.oracle.labs.mlrg.olcut.config.Options
CLI Options for creating a tokenizer.

  • Field Summary

    Fields inherited from interface com.oracle.labs.mlrg.olcut.config.Options

    header

  • Method Summary

    Modifier and Type
    Method
    Description
    Tokenizer
    getTokenizer()
    Creates the appropriately configured tokenizer.

    Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Options

    getOptionsDescription

  • Method Details

    • getTokenizer

      Tokenizer getTokenizer()
      Creates the appropriately configured tokenizer.
      Returns:
      The configured tokenizer.