Package org.tribuo.data.text.impl

package org.tribuo.data.text.impl

Provides implementations of text data processors.

Related Packages

Package	Description
org.tribuo.data.text	Provides interfaces for converting text inputs into Features and Examples.

Class	Description
AverageAggregator	A feature aggregator that averages feature values across a feature list.
BasicPipeline	An example implementation of TextPipeline.
CasingPreprocessor	A document preprocessor which uppercases or lowercases the input.
CasingPreprocessor.CasingOperation	The possible casing operations.
FeatureHasher	Hashes the feature names to reduce the dimensionality.
NewsPreprocessor	A document pre-processor for 20 newsgroup data.
NgramProcessor	A text processor that will generate token ngrams of a particular size.
RegexPreprocessor	A simple document preprocessor which applies regular expressions to the input.
SimpleStringDataSource<T extends Output<T>>	A version of SimpleTextDataSource that accepts a List of Strings.
SimpleStringDataSource.SimpleStringDataSourceProvenance	Provenance for SimpleStringDataSource.
SimpleTextDataSource<T extends Output<T>>	A dataset for a simple data format for text classification experiments.
SimpleTextDataSource.SimpleTextDataSourceProvenance	Provenance for SimpleTextDataSource.
SumAggregator	A feature aggregator that aggregates occurrence counts across a number of feature lists.
TextFeatureExtractorImpl<T extends Output<T>>	An implementation of TextFeatureExtractor that takes a TextPipeline and generates ArrayExample.
TokenPipeline	A pipeline for generating ngram features.
UniqueAggregator	Aggregates feature tokens, generating unique features.