Enum Token.TokenType

java.lang.Object
java.lang.Enum<Token.TokenType>
org.tribuo.util.tokens.Token.TokenType
All Implemented Interfaces:
Serializable, Comparable<Token.TokenType>, java.lang.constant.Constable
Enclosing class:
Token

public static enum Token.TokenType extends Enum<Token.TokenType>
Tokenizers may product multiple kinds of tokens, depending on the application to which they're being put. For example, when processing a document for highlighting during querying, we need to send through whitespace and punctuation so that the document looks as it did in it's original form. For most tokenizer applications, they will only send word tokens.
  • Enum Constant Details

  • Method Details

    • values

      public static Token.TokenType[] values()
      Returns an array containing the constants of this enum type, in the order they are declared.
      Returns:
      an array containing the constants of this enum type, in the order they are declared
    • valueOf

      public static Token.TokenType valueOf(String name)
      Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)
      Parameters:
      name - the name of the enum constant to be returned.
      Returns:
      the enum constant with the specified name
      Throws:
      IllegalArgumentException - if this enum type has no constant with the specified name
      NullPointerException - if the argument is null