Class ParameterAveraging

All Implemented Interfaces:,<>, StochasticGradientOptimiser

public class ParameterAveraging extends Object implements StochasticGradientOptimiser
Averages the parameters across a gradient run.

Wraps an inner gradient optimiser. Only changes the values when finalise() is called


 Polyak BT, Juditsky AB
 "Acceleration of Stochastic Approximation by Averaging"
 SIAM Journal on Control and Optimization, 1992.
  • Constructor Details

    • ParameterAveraging

      public ParameterAveraging(StochasticGradientOptimiser optimiser)
      Adds parameter averaging around a gradient optimiser.
      optimiser - The inner optimiser to use to scale the gradients.
  • Method Details

    • initialise

      public void initialise(Parameters parameters)
      Description copied from interface: StochasticGradientOptimiser
      Initialises the gradient optimiser.

      Configures any learning rate parameters.

      Specified by:
      initialise in interface StochasticGradientOptimiser
      parameters - The parameters to optimise.
    • step

      public Tensor[] step(Tensor[] updates, double weight)
      This passes the gradient update to the inner optimiser, then updates the average weight values.
      Specified by:
      step in interface StochasticGradientOptimiser
      updates - An array of gradients.
      weight - The weight for the current gradients.
      The gradients from the inner optimiser.
    • finalise

      public void finalise()
      This sets the parameters to their average value.
      Specified by:
      finalise in interface StochasticGradientOptimiser
    • toString

      public String toString()
      toString in class Object
    • reset

      public void reset()
      Description copied from interface: StochasticGradientOptimiser
      Resets the optimiser so it's ready to optimise a new Parameters.
      Specified by:
      reset in interface StochasticGradientOptimiser
    • copy

      public ParameterAveraging copy()
      Description copied from interface: StochasticGradientOptimiser
      Copies a gradient optimiser with it's configuration. Usually calls the copy constructor.
      Specified by:
      copy in interface StochasticGradientOptimiser
      A gradient optimiser with the same configuration, but independent state.
    • getProvenance

      public getProvenance()
      Specified by:
      getProvenance in interface<>