Package org.tribuo.data.text

Provides interfaces for converting text inputs into Features and Examples.

See: Description

Interface Summary

Interface	Description
DocumentPreprocessor	An interface for things that can pre-process documents before they are broken into features.
FeatureAggregator	An interface for aggregating feature values into other values.
FeatureTransformer	A feature transformer maps a list of features to a new list of features Useful for example to apply the hashing trick to a set of features
TextFeatureExtractor<T extends Output<T>>	An interface for things that take text and turn them into examples that we can use to train or evaluate a classifier.
TextPipeline	A pipeline that takes a String and returns a List of Features.
TextProcessor	A TextProcessor takes some text and optionally a feature tag and generates a list of Features from that text.

Class Summary

Class	Description
DirectoryFileSource<T extends Output<T>>	A data source for a somewhat-common format for text classification datasets: a top level directory that contains a number of subdirectories.
DirectoryFileSource.DirectoryFileSourceProvenance	Provenance for DirectoryFileSource.
SplitTextData	Splits data in our standard text format into training and testing portions.
SplitTextData.TrainTestSplitOptions	Command line options.
TextDataSource<T extends Output<T>>	A base class for textual data sets.

Exception Summary

Exception	Description
TextProcessingException	An exception thrown by the text processing system.

Package org.tribuo.data.text Description

Provides interfaces for converting text inputs into Features and Examples.