Class MinimumCardinalitySequenceDataset<T extends Output<T>>

java.lang.Object
org.tribuo.sequence.SequenceDataset<T>
org.tribuo.sequence.ImmutableSequenceDataset<T>
org.tribuo.sequence.MinimumCardinalitySequenceDataset<T>
Type Parameters:
T - The type of the outputs in this SequenceDataset.
All Implemented Interfaces:
com.oracle.labs.mlrg.olcut.provenance.Provenancable<DatasetProvenance>, Serializable, Iterable<SequenceExample<T>>

This class creates a pruned dataset in which low frequency features that occur less than the provided minimum cardinality have been removed. This can be useful when the dataset is very large due to many low-frequency features. Here, a new dataset is created so that the feature counts are recalculated and so that the original, passed-in dataset is not modified. The returned dataset may have fewer sequence examples because if any of the sequence examples have examples with no features after the minimum cardinality has been applied, then those sequence examples will not be added to the constructed dataset.
See Also:
  • Constructor Details

    • MinimumCardinalitySequenceDataset

      public MinimumCardinalitySequenceDataset(SequenceDataset<T> sequenceDataset, int minCardinality)
      Parameters:
      sequenceDataset - this dataset is left untouched and is used to populate the constructed dataset.
      minCardinality - features with a frequency less than minCardinality will be removed.
  • Method Details