org.tribuo.sequence

Class MutableSequenceDataset<T extends Output<T>>

java.lang.Object

org.tribuo.sequence.SequenceDataset<T>

org.tribuo.sequence.MutableSequenceDataset<T>

All Implemented Interfaces:

com.oracle.labs.mlrg.olcut.provenance.Provenancable<DatasetProvenance>, Serializable, Iterable<SequenceExample<T>>

public class MutableSequenceDataset<T extends Output<T>>
extends SequenceDataset<T>

A MutableSequenceDataset is a SequenceDataset with a MutableFeatureMap which grows over time. Whenever an SequenceExample is added to the dataset.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	dense
protected MutableFeatureMap	featureMap A map from feature names to IDs for the features found in this dataset.
protected MutableOutputInfo<T>	outputInfo A map from labels to IDs for the labels found in this dataset.

Fields inherited from class org.tribuo.sequence.SequenceDataset

data, outputFactory, sourceProvenance

Constructor Summary

Constructors

Constructor and Description
MutableSequenceDataset(DataProvenance sourceProvenance, OutputFactory<T> outputFactory) Creates an empty sequence dataset.
MutableSequenceDataset(ImmutableSequenceDataset<T> dataset)
MutableSequenceDataset(Iterable<SequenceExample<T>> dataSource, DataProvenance sourceProvenance, OutputFactory<T> outputFactory) Creates a dataset from a data source.
MutableSequenceDataset(SequenceDataSource<T> dataSource)

Method Summary

Modifier and Type	Method and Description
void	add(SequenceExample<T> ex) Adds a SequenceExample to this dataset.
void	addAll(Collection<SequenceExample<T>> collection) Adds all the SequenceExamples in the supplied collection to this dataset.
void	clear() Clears all the examples out of this dataset, and flushes the FeatureMap, OutputInfo, and transform provenances.
void	densify() Iterates through the examples, converting implicit zeros into explicit zeros.
ImmutableFeatureMap	getFeatureIDMap() An immutable view on the feature map.
MutableFeatureMap	getFeatureMap() The feature map.
ImmutableOutputInfo<T>	getOutputIDInfo() An immutable view on the output info in this dataset.
OutputInfo<T>	getOutputInfo() The output info in this dataset.
Set<T>	getOutputs() Gets the set of labels that occur in the examples in this dataset.
DatasetProvenance	getProvenance()
boolean	isDense() Is the dataset dense (i.e., do all features in the domain have a value in each example).
String	toString()

Methods inherited from class org.tribuo.sequence.SequenceDataset

getData, getExample, getFlatDataset, getOutputFactory, getSourceDescription, getSourceProvenance, iterator, size

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

outputInfo

protected final MutableOutputInfo<T extends Output<T>> outputInfo

A map from labels to IDs for the labels found in this dataset.

featureMap

protected final MutableFeatureMap featureMap

A map from feature names to IDs for the features found in this dataset.

dense

protected boolean dense

Constructor Detail

MutableSequenceDataset

public MutableSequenceDataset(DataProvenance sourceProvenance,
                              OutputFactory<T> outputFactory)

Creates an empty sequence dataset.

Parameters:

sourceProvenance - A description of the input data, including preprocessing steps.

outputFactory - The output factory.

MutableSequenceDataset

public MutableSequenceDataset(Iterable<SequenceExample<T>> dataSource,
                              DataProvenance sourceProvenance,
                              OutputFactory<T> outputFactory)

Creates a dataset from a data source. This method will create the output and feature ID maps that are needed for training and evaluating classifiers.

Parameters:

dataSource - The input data.

sourceProvenance - A description of the data, including preprocessing steps.

outputFactory - The output factory.

MutableSequenceDataset

public MutableSequenceDataset(SequenceDataSource<T> dataSource)

MutableSequenceDataset

public MutableSequenceDataset(ImmutableSequenceDataset<T> dataset)

Method Detail

clear

public void clear()

Clears all the examples out of this dataset, and flushes the FeatureMap, OutputInfo, and transform provenances.

add

public void add(SequenceExample<T> ex)

Adds a SequenceExample to this dataset.

It also canonicalises the reference to each feature's name (i.e., replacing the reference to a feature's name with the canonical one stored in this Dataset's VariableInfo). This greatly reduces the memory footprint.

Parameters:

ex - The example to add.

addAll

public void addAll(Collection<SequenceExample<T>> collection)

Adds all the SequenceExamples in the supplied collection to this dataset.

Parameters:

collection - The collection of SequenceExamples.

getOutputs

public Set<T> getOutputs()

Description copied from class: SequenceDataset

Gets the set of labels that occur in the examples in this dataset.

Specified by:

getOutputs in class SequenceDataset<T extends Output<T>>

Returns:

the set of labels that occur in the examples in this dataset.

getFeatureIDMap

public ImmutableFeatureMap getFeatureIDMap()

Description copied from class: SequenceDataset

An immutable view on the feature map.

Specified by:

getFeatureIDMap in class SequenceDataset<T extends Output<T>>

Returns:

The feature map.

getFeatureMap

public MutableFeatureMap getFeatureMap()

Description copied from class: SequenceDataset

The feature map.

Specified by:

getFeatureMap in class SequenceDataset<T extends Output<T>>

Returns:

The feature map.

getOutputIDInfo

public ImmutableOutputInfo<T> getOutputIDInfo()

Description copied from class: SequenceDataset

An immutable view on the output info in this dataset.

Specified by:

getOutputIDInfo in class SequenceDataset<T extends Output<T>>

Returns:

The output info.

getOutputInfo

public OutputInfo<T> getOutputInfo()

Description copied from class: SequenceDataset

The output info in this dataset.

Specified by:

getOutputInfo in class SequenceDataset<T extends Output<T>>

Returns:

The output info.

isDense

public boolean isDense()

Is the dataset dense (i.e., do all features in the domain have a value in each example).

Returns:

True if the dataset is dense.

densify

public void densify()

Iterates through the examples, converting implicit zeros into explicit zeros.

toString

public String toString()

Overrides:

toString in class SequenceDataset<T extends Output<T>>

getProvenance

public DatasetProvenance getProvenance()