Interface TokenizerOptions

All Superinterfaces:
com.oracle.labs.mlrg.olcut.config.Options
All Known Implementing Classes:
BreakIteratorTokenizerOptions, CoreTokenizerOptions, SplitCharactersTokenizerOptions, SplitPatternTokenizerOptions

public interface TokenizerOptions extends com.oracle.labs.mlrg.olcut.config.Options
CLI Options for creating a tokenizer.
  • Field Summary

    Fields inherited from interface com.oracle.labs.mlrg.olcut.config.Options

    header
  • Method Summary

    Modifier and Type
    Method
    Description
    Creates the appropriately configured tokenizer.

    Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Options

    getOptionsDescription
  • Method Details

    • getTokenizer

      Tokenizer getTokenizer()
      Creates the appropriately configured tokenizer.
      Returns:
      The configured tokenizer.