org.tribuo.util.tokens.impl

Interface SplitFunctionTokenizer.SplitFunction

All Known Implementing Classes:

SplitCharactersTokenizer.SplitCharactersSplitterFunction

Enclosing class:

SplitFunctionTokenizer

Functional Interface:

This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.

@FunctionalInterface
public static interface SplitFunctionTokenizer.SplitFunction

An interface for checking if the text should be split at the supplied codepoint.

Method Summary

Modifier and Type	Method and Description
SplitFunctionTokenizer.SplitResult	apply(int codepoint, int index, CharSequence cs) Applies the split function.

Method Detail

apply

SplitFunctionTokenizer.SplitResult apply(int codepoint,
                                         int index,
                                         CharSequence cs)

Applies the split function.

Parameters:

codepoint - The codepoint to check.

index - The character index.

cs - The sequence that's being split.

Returns:

How the sequence should be split.