Class AdaGradRDA

java.lang.Object
org.tribuo.math.optimisers.AdaGradRDA
All Implemented Interfaces:
com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>, StochasticGradientOptimiser
public class AdaGradRDA extends Object implements StochasticGradientOptimiser
An implementation of the AdaGrad gradient optimiser with regularized dual averaging.

This gradient optimiser rewrites all the Tensors in the Parameters with AdaGradRDA.AdaGradRDATensor. This means it keeps a different value in the Tensor to the one produced when you call get(), so it can correctly apply regularisation to the parameters. When finalise() is called it rewrites the Parameters with standard dense Tensors. Follows the implementation in Factorie.

See: 

 Duchi, J., Hazan, E., and Singer, Y.
 "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization"
 Journal of Machine Learning Research, 2012, 2121-2159.

  • Constructor Summary

    Constructors
    Constructor
    Description
    AdaGradRDA(double initialLearningRate, double epsilon)
    Creates an AdaGradRDA optimiser with the specified parameter values.
    AdaGradRDA(double initialLearningRate, double epsilon, double l1, double l2, int numExamples)
    Creates an AdaGradRDA optimiser with the specified parameter values.

  • Method Summary

    Modifier and Type
    Method
    Description
    AdaGradRDA
    copy()
    Copies a gradient optimiser with it's configuration.
    void
    finalise()
    Finalises the gradient optimisation, setting the parameters to their correct values.
    com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance
    getProvenance()
     
    void
    initialise(Parameters parameters)
    Initialises the gradient optimiser.
    void
    reset()
    Resets the optimiser so it's ready to optimise a new Parameters.
    Tensor[]
    step(Tensor[] updates, double weight)
    Take a Tensor array of gradients and transform them according to the current weight and learning rates.
    String
    toString()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Configurable

    postConfig

  • Constructor Details

    • AdaGradRDA

      public AdaGradRDA(double initialLearningRate, double epsilon, double l1, double l2, int numExamples)
      Creates an AdaGradRDA optimiser with the specified parameter values.
      Parameters:
      initialLearningRate - The learning rate.
      epsilon - The epsilon value for stabilising the gradient inversion.
      l1 - The l1 penalty.
      l2 - The l2 penalty.
      numExamples - The number of examples to scale the l1 and l2 penalties.

    • AdaGradRDA

      public AdaGradRDA(double initialLearningRate, double epsilon)
      Creates an AdaGradRDA optimiser with the specified parameter values.

      Sets the regularisation parameters to zero.

      Parameters:
      initialLearningRate - The learning rate.
      epsilon - The epsilon value for stabilising the gradient inversion.

  • Method Details