public class AdaDelta extends Object implements StochasticGradientOptimiser
Creates two copies of the parameters to store learning rates.
See:
Zeiler, MD. "ADADELTA: an Adaptive Learning Rate Method" arXiv preprint arXiv:1212.5701.
Constructor and Description |
---|
AdaDelta()
Sets rho to 0.95 and epsilon to 1e-6.
|
AdaDelta(double epsilon)
Keeps rho at 0.95, passes through epsilon.
|
AdaDelta(double rho,
double epsilon)
It's recommended to keep rho at 0.95.
|
Modifier and Type | Method and Description |
---|---|
AdaDelta |
copy()
Copies a gradient optimiser with it's configuration.
|
com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance |
getProvenance() |
void |
initialise(Parameters parameters)
Initialises the gradient optimiser.
|
void |
reset()
Resets the optimiser so it's ready to optimise a new
Parameters . |
Tensor[] |
step(Tensor[] updates,
double weight)
Take a
Tensor array of gradients and transform them
according to the current weight and learning rates. |
String |
toString() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
finalise
public AdaDelta(double rho, double epsilon)
rho
- The rho value.epsilon
- The epsilon value.public AdaDelta(double epsilon)
epsilon
- The epsilon value.public AdaDelta()
public void initialise(Parameters parameters)
StochasticGradientOptimiser
Configures any learning rate parameters.
initialise
in interface StochasticGradientOptimiser
parameters
- The parameters to optimise.public Tensor[] step(Tensor[] updates, double weight)
StochasticGradientOptimiser
Tensor
array of gradients and transform them
according to the current weight and learning rates.
Can return the same Tensor
array or a new one.
step
in interface StochasticGradientOptimiser
updates
- An array of gradients.weight
- The weight for the current gradients.Tensor
array of gradients.public void reset()
StochasticGradientOptimiser
Parameters
.reset
in interface StochasticGradientOptimiser
public AdaDelta copy()
StochasticGradientOptimiser
copy
in interface StochasticGradientOptimiser
public com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance getProvenance()
getProvenance
in interface com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>
Copyright © 2015–2021 Oracle and/or its affiliates. All rights reserved.