org.tribuo.math.optimisers

Class AdaGradRDA

java.lang.Object

org.tribuo.math.optimisers.AdaGradRDA

All Implemented Interfaces:

com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>, StochasticGradientOptimiser

public class AdaGradRDA
extends Object
implements StochasticGradientOptimiser

An implementation of the AdaGrad gradient optimiser with regularized dual averaging.

This gradient optimiser rewrites all the Tensors in the Parameters with AdaGradRDATensor. This means it keeps a different value in the Tensor to the one produced when you call get(), so it can correctly apply regularisation to the parameters. When finalise() is called it rewrites the Parameters with standard dense Tensors. Follows the implementation in Factorie.

See:

 Duchi, J., Hazan, E., and Singer, Y.
 "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization"
 Journal of Machine Learning Research, 2012, 2121-2159.
 

Constructor Summary

Constructors

Constructor and Description
AdaGradRDA(double initialLearningRate, double epsilon)
AdaGradRDA(double initialLearningRate, double epsilon, double l1, double l2, int numExamples)

Method Summary

Modifier and Type	Method and Description
AdaGradRDA	copy() Copies a gradient optimiser with it's configuration.
void	finalise() Finalises the gradient optimisation, setting the parameters to their correct values.
com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance	getProvenance()
void	initialise(Parameters parameters) Initialises the gradient optimiser.
void	reset() Resets the optimiser so it's ready to optimise a new Parameters.
Tensor[]	step(Tensor[] updates, double weight) Take a Tensor array of gradients and transform them according to the current weight and learning rates.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Configurable

postConfig

Constructor Detail

AdaGradRDA

public AdaGradRDA(double initialLearningRate,
                  double epsilon,
                  double l1,
                  double l2,
                  int numExamples)

AdaGradRDA

public AdaGradRDA(double initialLearningRate,
                  double epsilon)

Method Detail

initialise

public void initialise(Parameters parameters)

Description copied from interface: StochasticGradientOptimiser

Initialises the gradient optimiser.

Configures any learning rate parameters.

Specified by:

initialise in interface StochasticGradientOptimiser

Parameters:

parameters - The parameters to optimise.

step

public Tensor[] step(Tensor[] updates,
                     double weight)

Description copied from interface: StochasticGradientOptimiser

Take a Tensor array of gradients and transform them according to the current weight and learning rates.

Can return the same Tensor array or a new one.

Specified by:

step in interface StochasticGradientOptimiser

Parameters:

updates - An array of gradients.

weight - The weight for the current gradients.

Returns:

A Tensor array of gradients.

finalise

public void finalise()

Description copied from interface: StochasticGradientOptimiser

Finalises the gradient optimisation, setting the parameters to their correct values. Used for ParameterAveraging amongst others.

Specified by:

finalise in interface StochasticGradientOptimiser

toString

public String toString()

Overrides:

toString in class Object

reset

public void reset()

Description copied from interface: StochasticGradientOptimiser

Resets the optimiser so it's ready to optimise a new Parameters.

Specified by:

reset in interface StochasticGradientOptimiser

copy

public AdaGradRDA copy()

Description copied from interface: StochasticGradientOptimiser

Copies a gradient optimiser with it's configuration. Usually calls the copy constructor.

Specified by:

copy in interface StochasticGradientOptimiser

Returns:

A gradient optimiser with the same configuration, but independent state.

getProvenance

public com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance getProvenance()

Specified by:

getProvenance in interface com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>