org.tribuo.math.optimisers

Class ParameterAveraging

java.lang.Object

org.tribuo.math.optimisers.ParameterAveraging

All Implemented Interfaces:

com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>, StochasticGradientOptimiser

public class ParameterAveraging
extends Object
implements StochasticGradientOptimiser

Averages the parameters across a gradient run.

Wraps an inner gradient optimiser. Only changes the values when finalise() is called

See:

 Polyak BT, Juditsky AB
 "Acceleration of Stochastic Approximation by Averaging"
 SIAM Journal on Control and Optimization, 1992.
 

Constructor Summary

Constructors

Constructor and Description
ParameterAveraging(StochasticGradientOptimiser optimiser) Adds parameter averaging around a gradient optimiser.

Method Summary

Modifier and Type	Method and Description
ParameterAveraging	copy() Copies a gradient optimiser with it's configuration.
void	finalise() This sets the parameters to their average value.
com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance	getProvenance()
void	initialise(Parameters parameters) Initialises the gradient optimiser.
void	reset() Resets the optimiser so it's ready to optimise a new Parameters.
Tensor[]	step(Tensor[] updates, double weight) This passes the gradient update to the inner optimiser, then updates the average weight values.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Configurable

postConfig

Constructor Detail

ParameterAveraging

public ParameterAveraging(StochasticGradientOptimiser optimiser)

Adds parameter averaging around a gradient optimiser.

Parameters:

optimiser - The inner optimiser to use to scale the gradients.

Method Detail

initialise

public void initialise(Parameters parameters)

Description copied from interface: StochasticGradientOptimiser

Initialises the gradient optimiser.

Configures any learning rate parameters.

Specified by:

initialise in interface StochasticGradientOptimiser

Parameters:

parameters - The parameters to optimise.

step

public Tensor[] step(Tensor[] updates,
                     double weight)

This passes the gradient update to the inner optimiser, then updates the average weight values.

Specified by:

step in interface StochasticGradientOptimiser

Parameters:

updates - An array of gradients.

weight - The weight for the current gradients.

Returns:

The gradients from the inner optimiser.

finalise

public void finalise()

This sets the parameters to their average value.

Specified by:

finalise in interface StochasticGradientOptimiser

toString

public String toString()

Overrides:

toString in class Object

reset

public void reset()

Description copied from interface: StochasticGradientOptimiser

Resets the optimiser so it's ready to optimise a new Parameters.

Specified by:

reset in interface StochasticGradientOptimiser

copy

public ParameterAveraging copy()

Description copied from interface: StochasticGradientOptimiser

Copies a gradient optimiser with it's configuration. Usually calls the copy constructor.

Specified by:

copy in interface StochasticGradientOptimiser

Returns:

A gradient optimiser with the same configuration, but independent state.

getProvenance

public com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance getProvenance()

Specified by:

getProvenance in interface com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>