Class Adam

java.lang.Object
org.tribuo.math.optimisers.Adam
All Implemented Interfaces:
com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>, StochasticGradientOptimiser
public class Adam extends Object implements StochasticGradientOptimiser
An implementation of the Adam gradient optimiser.

Creates two copies of the parameters to store learning rates.

See: 

 Kingma, D., and Ba, J.
 "Adam: A Method for Stochastic Optimization"
 arXiv preprint arXiv:1412.6980, 2014.

  • Constructor Summary

    Constructors
    Constructor
    Description
    Adam()
    Sets initialLearningRate to 0.001, betaOne to 0.9, betaTwo to 0.999, epsilon to 1e-6.
    Adam(double initialLearningRate, double epsilon)
    Sets betaOne to 0.9 and betaTwo to 0.999
    Adam(double initialLearningRate, double betaOne, double betaTwo, double epsilon)
    It's highly recommended not to modify these parameters, use one of the other constructors.

  • Method Summary

    Modifier and Type
    Method
    Description
    Adam
    copy()
    Copies a gradient optimiser with it's configuration.
    com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance
    getProvenance()
     
    void
    initialise(Parameters parameters)
    Initialises the gradient optimiser.
    void
    reset()
    Resets the optimiser so it's ready to optimise a new Parameters.
    Tensor[]
    step(Tensor[] updates, double weight)
    Take a Tensor array of gradients and transform them according to the current weight and learning rates.
    String
    toString()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Configurable

    postConfig

    Methods inherited from interface org.tribuo.math.StochasticGradientOptimiser

    finalise

  • Constructor Details

    • Adam

      public Adam(double initialLearningRate, double betaOne, double betaTwo, double epsilon)
      It's highly recommended not to modify these parameters, use one of the other constructors.
      Parameters:
      initialLearningRate - The initial learning rate.
      betaOne - The value of beta-one.
      betaTwo - The value of beta-two.
      epsilon - The epsilon value.

    • Adam

      public Adam(double initialLearningRate, double epsilon)
      Sets betaOne to 0.9 and betaTwo to 0.999
      Parameters:
      initialLearningRate - The initial learning rate.
      epsilon - The epsilon value.

    • Adam

      public Adam()
      Sets initialLearningRate to 0.001, betaOne to 0.9, betaTwo to 0.999, epsilon to 1e-6. These are the parameters from the Adam paper.

  • Method Details