org.tribuo.common.liblinear

Class LibLinearTrainer<T extends Output<T>>

java.lang.Object

org.tribuo.common.liblinear.LibLinearTrainer<T>

All Implemented Interfaces:

com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<TrainerProvenance>, Trainer<T>

Direct Known Subclasses:

LibLinearAnomalyTrainer, LibLinearClassificationTrainer, LibLinearRegressionTrainer

public abstract class LibLinearTrainer<T extends Output<T>>
extends Object
implements Trainer<T>

A Trainer which wraps a liblinear-java trainer.

Note the train method is synchronized on LibLinearTrainer.class due to a global RNG in liblinear-java. This is insufficient to ensure reproducibility if liblinear-java is used directly in the same JVM as Tribuo, but avoids locking on classes Tribuo does not control.

See:

 Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ.
 "LIBLINEAR: A library for Large Linear Classification"
 Journal of Machine Learning Research, 2008.
 

and for the original algorithm:

 Cortes C, Vapnik V.
 "Support-Vector Networks"
 Machine Learning, 1995.
 

Field Summary

Fields

Modifier and Type	Field and Description
protected double	cost
protected double	epsilon
protected de.bwaldvogel.liblinear.Parameter	libLinearParams
protected int	maxIterations
protected double	terminationCriterion
protected LibLinearType<T>	trainerType

Fields inherited from interface org.tribuo.Trainer

DEFAULT_SEED

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	LibLinearTrainer()
protected	LibLinearTrainer(LibLinearType<T> trainerType, double cost, int maxIterations, double terminationCriterion) Creates a trainer for a LibLinear model
protected	LibLinearTrainer(LibLinearType<T> trainerType, double cost, int maxIterations, double terminationCriterion, double epsilon) Creates a trainer for a LibLinear model

Method Summary

Modifier and Type	Method and Description
protected abstract LibLinearModel<T>	createModel(ModelProvenance provenance, ImmutableFeatureMap featureIDMap, ImmutableOutputInfo<T> outputIDInfo, List<de.bwaldvogel.liblinear.Model> models) Construct the appropriate subtype of LibLinearModel for the prediction task.
static <T extends Output<T>> de.bwaldvogel.liblinear.FeatureNode[]	exampleToNodes(Example<T> example, ImmutableFeatureMap featureIDMap, List<de.bwaldvogel.liblinear.FeatureNode> features) Converts a Tribuo Example into a liblinear FeatureNode array, including a bias feature.
protected abstract com.oracle.labs.mlrg.olcut.util.Pair<de.bwaldvogel.liblinear.FeatureNode[][],double[][]>	extractData(Dataset<T> data, ImmutableOutputInfo<T> outputInfo, ImmutableFeatureMap featureMap) Extracts the features and Outputs in LibLinear's format.
int	getInvocationCount() The number of times this trainer instance has had it's train method invoked.
TrainerProvenance	getProvenance()
void	postConfig() Used by the OLCUT configuration system, and should not be called by external code.
protected de.bwaldvogel.liblinear.Parameter	setupParameters(ImmutableOutputInfo<T> info) Constructs the parameters.
String	toString()
LibLinearModel<T>	train(Dataset<T> examples) Trains a predictive model using the examples in the given data set.
LibLinearModel<T>	train(Dataset<T> examples, Map<String,com.oracle.labs.mlrg.olcut.provenance.Provenance> runProvenance) Trains a predictive model using the examples in the given data set.
protected abstract List<de.bwaldvogel.liblinear.Model>	trainModels(de.bwaldvogel.liblinear.Parameter curParams, int numFeatures, de.bwaldvogel.liblinear.FeatureNode[][] features, double[][] outputs) Train all the liblinear instances necessary for this dataset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

libLinearParams

protected de.bwaldvogel.liblinear.Parameter libLinearParams

trainerType

@Config(description="Algorithm to use.")
protected LibLinearType<T extends Output<T>> trainerType

cost

@Config(description="Cost penalty for misclassifications.")
protected double cost

maxIterations

@Config(description="Maximum number of iterations before terminating.")
protected int maxIterations

terminationCriterion

@Config(description="Stop iterating when the loss score decreases by less than this value.")
protected double terminationCriterion

epsilon

@Config(description="Epsilon insensitivity in the regression cost function.")
protected double epsilon

Constructor Detail

LibLinearTrainer

protected LibLinearTrainer()

LibLinearTrainer

protected LibLinearTrainer(LibLinearType<T> trainerType,
                           double cost,
                           int maxIterations,
                           double terminationCriterion)

Creates a trainer for a LibLinear model

Parameters:

trainerType - Loss function and optimisation method combination.

cost - Cost penalty for each incorrectly classified training point.

maxIterations - The maximum number of dataset iterations.

terminationCriterion - How close does the optimisation function need to be before terminating that subproblem (usually set to 0.1).

LibLinearTrainer

protected LibLinearTrainer(LibLinearType<T> trainerType,
                           double cost,
                           int maxIterations,
                           double terminationCriterion,
                           double epsilon)

Creates a trainer for a LibLinear model

Parameters:

trainerType - Loss function and optimisation method combination.

cost - Cost penalty for each incorrectly classified training point.

maxIterations - The maximum number of dataset iterations.

terminationCriterion - How close does the optimisation function need to be before terminating that subproblem (usually set to 0.1).

epsilon - The insensitivity of the regression loss to small differences.

Method Detail

postConfig

public void postConfig()

Used by the OLCUT configuration system, and should not be called by external code.

Specified by:

postConfig in interface com.oracle.labs.mlrg.olcut.config.Configurable

train

public LibLinearModel<T> train(Dataset<T> examples)

Description copied from interface: Trainer

Trains a predictive model using the examples in the given data set.

Specified by:

train in interface Trainer<T extends Output<T>>

Parameters:

examples - the data set containing the examples.

Returns:

a predictive model that can be used to generate predictions for new examples.

train

public LibLinearModel<T> train(Dataset<T> examples,
                               Map<String,com.oracle.labs.mlrg.olcut.provenance.Provenance> runProvenance)

Description copied from interface: Trainer

Trains a predictive model using the examples in the given data set.

Specified by:

train in interface Trainer<T extends Output<T>>

Parameters:

examples - the data set containing the examples.

runProvenance - Training run specific provenance (e.g., fold number).

Returns:

a predictive model that can be used to generate predictions for new examples.

getInvocationCount

public int getInvocationCount()

Description copied from interface: Trainer

The number of times this trainer instance has had it's train method invoked.

This is used to determine how many times the trainer's RNG has been accessed to ensure replicability in the random number stream.

Specified by:

getInvocationCount in interface Trainer<T extends Output<T>>

Returns:

The number of train invocations.

toString

public String toString()

Overrides:

toString in class Object

trainModels

protected abstract List<de.bwaldvogel.liblinear.Model> trainModels(de.bwaldvogel.liblinear.Parameter curParams,
                                                                   int numFeatures,
                                                                   de.bwaldvogel.liblinear.FeatureNode[][] features,
                                                                   double[][] outputs)

Train all the liblinear instances necessary for this dataset.

Parameters:

curParams - The LibLinear parameters.

numFeatures - The number of features in this dataset.

features - The features themselves.

outputs - The outputs.

Returns:

A list of liblinear models.

createModel

protected abstract LibLinearModel<T> createModel(ModelProvenance provenance,
                                                 ImmutableFeatureMap featureIDMap,
                                                 ImmutableOutputInfo<T> outputIDInfo,
                                                 List<de.bwaldvogel.liblinear.Model> models)

Construct the appropriate subtype of LibLinearModel for the prediction task.

Parameters:

provenance - The model provenance.

featureIDMap - The feature id map.

outputIDInfo - The output id info.

models - The list of linear models.

Returns:

An implementation of LibLinearModel.

extractData

protected abstract com.oracle.labs.mlrg.olcut.util.Pair<de.bwaldvogel.liblinear.FeatureNode[][],double[][]> extractData(Dataset<T> data,
                                                                                                                        ImmutableOutputInfo<T> outputInfo,
                                                                                                                        ImmutableFeatureMap featureMap)

Extracts the features and Outputs in LibLinear's format.

Parameters:

data - The input data.

outputInfo - The output info.

featureMap - The feature info.

Returns:

The features and outputs.

setupParameters

protected de.bwaldvogel.liblinear.Parameter setupParameters(ImmutableOutputInfo<T> info)

Constructs the parameters. Most of the time this is a no-op, but classification overrides it to incorporate label weights if they exist.

Parameters:

info - The output info.

Returns:

The Parameters to use for training.

exampleToNodes

public static <T extends Output<T>> de.bwaldvogel.liblinear.FeatureNode[] exampleToNodes(Example<T> example,
                                                                                         ImmutableFeatureMap featureIDMap,
                                                                                         List<de.bwaldvogel.liblinear.FeatureNode> features)

Converts a Tribuo Example into a liblinear FeatureNode array, including a bias feature.

If there is a collision between feature ids (i.e., if there is feature hashing or some other mechanism changing the feature ids) then the feature values are summed.

Type Parameters:

T - The output type.

Parameters:

example - The input example.

featureIDMap - The feature id map which contains the example's indices.

features - A buffer. If null then an array list is created and used internally.

Returns:

The features suitable for use in liblinear.

getProvenance

public TrainerProvenance getProvenance()

Specified by:

getProvenance in interface com.oracle.labs.mlrg.olcut.provenance.Provenancable<TrainerProvenance>