Uses of Class
org.tribuo.util.tokens.impl.wordpiece.Wordpiece

Packages that use Wordpiece

Package	Description
org.tribuo.util.tokens.impl.wordpiece	Provides an implementation of a Wordpiece tokenizer which implements to the Tribuo Tokenizer API.

Uses of Wordpiece in org.tribuo.util.tokens.impl.wordpiece

Constructors in org.tribuo.util.tokens.impl.wordpiece with parameters of type Wordpiece

Constructor and Description
WordpieceTokenizer(Wordpiece wordpiece, Tokenizer tokenizer, boolean toLowerCase, boolean stripAccents, Set<String> neverSplit) Constructs a wordpiece tokenizer.