- Type Parameters:
T- The type of the outputs in this
- All Implemented Interfaces:
This class creates a pruned dataset in which low frequency features that occur less than the provided minimum cardinality have been removed. This can be useful when the dataset is very large due to many low-frequency features. For example, this class can be used to remove low frequency words from a BoW formatted dataset. Here, a new dataset is created so that the feature counts are recalculated and so that the original, passed-in dataset is not modified. The returned dataset may have fewer examples because if any of the examples have no features after the minimum cardinality has been applied, then those examples will not be added to the constructed dataset.
- See Also:
Nested Class SummaryModifier and TypeClassDescription
static classProvenance for
Fields inherited from class org.tribuo.ImmutableDataset
dropInvalidExamples, featureIDMap, outputIDInfo
Methods inherited from class org.tribuo.ImmutableDataset
add, add, copyDataset, copyDataset, copyDataset, getDropInvalidExamples, getFeatureIDMap, getFeatureMap, getOutputIDInfo, getOutputInfo, getOutputs, hashFeatureMap, toString
Methods inherited from class org.tribuo.Dataset
castDataset, createTransformers, createTransformers, getData, getExample, getOutputFactory, getSourceDescription, getSourceProvenance, iterator, shuffle, size, validate
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
dataset- this dataset is left untouched and is used to populate the constructed dataset.
minCardinality- features with a frequency less than minCardinality will be removed.
getRemovedThe feature names that were removed.
- The feature names.
getNumExamplesRemovedpublic int getNumExamplesRemoved()The number of examples removed due to a lack of features.
- The number of removed examples.
getMinCardinalitypublic int getMinCardinality()The minimum cardinality threshold for the features.
- The cardinality threshold.
getProvenancepublic DatasetProvenance getProvenance()