Package org.tribuo.util.tokens.impl

Enum Class SplitFunctionTokenizer.SplitType

java.lang.Object

java.lang.Enum<SplitFunctionTokenizer.SplitType>

org.tribuo.util.tokens.impl.SplitFunctionTokenizer.SplitType

All Implemented Interfaces:

Serializable, Comparable<SplitFunctionTokenizer.SplitType>, Constable

Enclosing class:

SplitFunctionTokenizer

public static enum SplitFunctionTokenizer.SplitType
extends Enum<SplitFunctionTokenizer.SplitType>

Defines different ways that a tokenizer can split the input text at a given character.

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Enum

Enum.EnumDesc<E extends Enum<E>>

Enum Constant Summary

Enum Constants

Enum Constant	Description
NO_SPLIT	the current character is added to the in-progress token (i.e.
SPLIT_AFTER	The current character will cause the in-progress token to be completed after the current character is appended to the in-progress token.
SPLIT_AT	The current character will cause the in-progress token to be completed.
SPLIT_BEFORE	The current character will cause the in-progress token to be completed the current character will be included in the next token.
SPLIT_BEFORE_AND_AFTER	The current character should cause the in-progress token to be completed.

Method Summary

Modifier and Type	Method	Description
static SplitFunctionTokenizer.SplitType	valueOf(String name)	Returns the enum constant of this class with the specified name.
static SplitFunctionTokenizer.SplitType[]	values()	Returns an array containing the constants of this enum class, in the order they are declared.

Methods inherited from class java.lang.Enum

clone, compareTo, describeConstable, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Enum Constant Details

NO_SPLIT

public static final SplitFunctionTokenizer.SplitType NO_SPLIT

the current character is added to the in-progress token (i.e. do not split on the current character)

SPLIT_AT

public static final SplitFunctionTokenizer.SplitType SPLIT_AT

The current character will cause the in-progress token to be completed. the current character will not be included in any returned token and the token type of the corresponding SplitResult is ignored (See SplitFunctionTokenizer.SplitResult.SPLIT_AT). This SplitType may be useful for whitespace.

SPLIT_BEFORE

public static final SplitFunctionTokenizer.SplitType SPLIT_BEFORE

The current character will cause the in-progress token to be completed the current character will be included in the next token. The token type of the corresponding SplitResult is ignored (See SplitFunctionTokenizer.SplitResult.SPLIT_BEFORE). This SplitType may be useful for e.g. capitalized letters when CamelCase splitting of digits when separating out a currency symbol.

SPLIT_AFTER

public static final SplitFunctionTokenizer.SplitType SPLIT_AFTER

The current character will cause the in-progress token to be completed after the current character is appended to the in-progress token. The token type of the created token (that includes the current character) will be assigned the type included with the SplitFunctionTokenizer.SplitResult.

SPLIT_BEFORE_AND_AFTER

public static final SplitFunctionTokenizer.SplitType SPLIT_BEFORE_AND_AFTER

The current character should cause the in-progress token to be completed. The token assigned to the in-progress token will be whatever was previously assigned to the previous character. This token will be followed by a second single-character token consisting of the current character. The token type assigned to this second token will be provided with the SplitFunctionTokenizer.SplitResult.

Method Details

values

public static SplitFunctionTokenizer.SplitType[] values()

Returns an array containing the constants of this enum class, in the order they are declared.

Returns:

an array containing the constants of this enum class, in the order they are declared

valueOf

public static SplitFunctionTokenizer.SplitType valueOf(String name)

Returns the enum constant of this class with the specified name. The string must match exactly an identifier used to declare an enum constant in this class. (Extraneous whitespace characters are not permitted.)

Parameters:

name - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum class has no constant with the specified name

NullPointerException - if the argument is null