org.tribuo.datasource

Class IDXDataSource<T extends Output<T>>

java.lang.Object

org.tribuo.datasource.IDXDataSource<T>

All Implemented Interfaces:

com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<DataSourceProvenance>, Iterable<Example<T>>, ConfigurableDataSource<T>, DataSource<T>

public final class IDXDataSource<T extends Output<T>>
extends Object
implements ConfigurableDataSource<T>

A DataSource which can read IDX formatted data (i.e., MNIST).

Transparently reads GZipped files.

The file format is defined here.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	IDXDataSource.IDXData Java side representation for an IDX file.
static class	IDXDataSource.IDXDataSourceProvenance Provenance class for IDXDataSource.
static class	IDXDataSource.IDXType The possible IDX input formats.

Constructor Summary

Constructors

Constructor and Description
IDXDataSource(Path featuresPath, Path outputPath, OutputFactory<T> outputFactory) Constructs an IDXDataSource from the supplied paths.

Method Summary

Modifier and Type	Method and Description
IDXDataSource.IDXType	getDataType() The type of the features that were loaded in.
OutputFactory<T>	getOutputFactory() Returns the OutputFactory associated with this Output subclass.
DataSourceProvenance	getProvenance()
Iterator<Example<T>>	iterator()
void	postConfig() Used by the OLCUT configuration system, and should not be called by external code.
int	size() The number of examples loaded.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

IDXDataSource

public IDXDataSource(Path featuresPath,
                     Path outputPath,
                     OutputFactory<T> outputFactory)
              throws IOException

Constructs an IDXDataSource from the supplied paths.

Parameters:

featuresPath - The path to the features file.

outputPath - The path to the output file.

outputFactory - The output factory.

Throws:

IOException - If either file cannot be read.

Method Detail

postConfig

public void postConfig()
                throws IOException

Used by the OLCUT configuration system, and should not be called by external code.

Specified by:

postConfig in interface com.oracle.labs.mlrg.olcut.config.Configurable

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

getOutputFactory

public OutputFactory<T> getOutputFactory()

Description copied from interface: DataSource

Returns the OutputFactory associated with this Output subclass.

Specified by:

getOutputFactory in interface DataSource<T extends Output<T>>

Returns:

The output factory.

getProvenance

public DataSourceProvenance getProvenance()

Specified by:

getProvenance in interface com.oracle.labs.mlrg.olcut.provenance.Provenancable<DataSourceProvenance>

size

public int size()

The number of examples loaded.

Returns:

The number of examples.

getDataType

public IDXDataSource.IDXType getDataType()

The type of the features that were loaded in.

Returns:

The feature type.

iterator

public Iterator<Example<T>> iterator()

Specified by:

iterator in interface Iterable<Example<T extends Output<T>>>