Class AdaDelta

java.lang.Object
org.tribuo.math.optimisers.AdaDelta
All Implemented Interfaces:
com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>, StochasticGradientOptimiser
public class AdaDelta extends Object implements StochasticGradientOptimiser
An implementation of the AdaDelta gradient optimiser.

Creates two copies of the parameters to store learning rates.

See: 

 Zeiler, MD.
 "ADADELTA: an Adaptive Learning Rate Method"
 arXiv preprint arXiv:1212.5701.

  • Constructor Summary

    Constructors
    Constructor
    Description
    AdaDelta()
    Sets rho to 0.95 and epsilon to 1e-6.
    AdaDelta(double epsilon)
    Keeps rho at 0.95, passes through epsilon.
    AdaDelta(double rho, double epsilon)
    It's recommended to keep rho at 0.95.

  • Method Summary

    Modifier and Type
    Method
    Description
    AdaDelta
    copy()
    Copies a gradient optimiser with it's configuration.
    com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance
    getProvenance()
     
    void
    initialise(Parameters parameters)
    Initialises the gradient optimiser.
    void
    reset()
    Resets the optimiser so it's ready to optimise a new Parameters.
    Tensor[]
    step(Tensor[] updates, double weight)
    Take a Tensor array of gradients and transform them according to the current weight and learning rates.
    String
    toString()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Configurable

    postConfig

    Methods inherited from interface org.tribuo.math.StochasticGradientOptimiser

    finalise

  • Constructor Details

    • AdaDelta

      public AdaDelta(double rho, double epsilon)
      It's recommended to keep rho at 0.95.
      Parameters:
      rho - The rho value.
      epsilon - The epsilon value.

    • AdaDelta

      public AdaDelta(double epsilon)
      Keeps rho at 0.95, passes through epsilon.
      Parameters:
      epsilon - The epsilon value.

    • AdaDelta

      public AdaDelta()
      Sets rho to 0.95 and epsilon to 1e-6.

  • Method Details