Class ParameterAveraging

java.lang.Object
org.tribuo.math.optimisers.ParameterAveraging
All Implemented Interfaces:
com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>, StochasticGradientOptimiser
public class ParameterAveraging extends Object implements StochasticGradientOptimiser
Averages the parameters across a gradient run.

Wraps an inner gradient optimiser. Only changes the values when finalise() is called

See: 

 Polyak BT, Juditsky AB
 "Acceleration of Stochastic Approximation by Averaging"
 SIAM Journal on Control and Optimization, 1992.

  • Constructor Details

    • ParameterAveraging

      public ParameterAveraging(StochasticGradientOptimiser optimiser)
      Adds parameter averaging around a gradient optimiser.
      Parameters:
      optimiser - The inner optimiser to use to scale the gradients.

  • Method Details

    • initialise

      public void initialise(Parameters parameters)
      Description copied from interface: StochasticGradientOptimiser
      Initialises the gradient optimiser.

      Configures any learning rate parameters.

      Specified by:
      initialise in interface StochasticGradientOptimiser
      Parameters:
      parameters - The parameters to optimise.

    • step

      public Tensor[] step(Tensor[] updates, double weight)
      This passes the gradient update to the inner optimiser, then updates the average weight values.
      Specified by:
      step in interface StochasticGradientOptimiser
      Parameters:
      updates - An array of gradients.
      weight - The weight for the current gradients.
      Returns:
      The gradients from the inner optimiser.

    • finalise

      public void finalise()
      This sets the parameters to their average value.
      Specified by:
      finalise in interface StochasticGradientOptimiser

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • reset

      public void reset()
      Description copied from interface: StochasticGradientOptimiser
      Resets the optimiser so it's ready to optimise a new Parameters.
      Specified by:
      reset in interface StochasticGradientOptimiser

    • copy

      public ParameterAveraging copy()
      Description copied from interface: StochasticGradientOptimiser
      Copies a gradient optimiser with it's configuration. Usually calls the copy constructor.
      Specified by:
      copy in interface StochasticGradientOptimiser
      Returns:
      A gradient optimiser with the same configuration, but independent state.

    • getProvenance

      public com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance getProvenance()
      Specified by:
      getProvenance in interface com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>