Package org.tribuo.math.optimisers

Class Pegasos

java.lang.Object
org.tribuo.math.optimisers.Pegasos
All Implemented Interfaces:
com.oracle.labs.mlrg.olcut.config.Configurable, com.oracle.labs.mlrg.olcut.provenance.Provenancable<com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance>, StochasticGradientOptimiser
public class Pegasos extends Object implements StochasticGradientOptimiser
An implementation of the Pegasos gradient optimiser used primarily for solving the SVM problem.

This gradient optimiser rewrites all the Tensors in the Parameters with ShrinkingTensor. This means it keeps a different value in the Tensor to the one produced when you call get(), so it can correctly apply regularisation to the parameters. When finalise() is called it rewrites the Parameters with standard dense Tensors. Follows the implementation in Factorie.

Pegasos is remarkably touchy about it's learning rates. The defaults work on a couple of examples, but it requires tuning to work properly on a specific dataset.

See: 

 Shalev-Shwartz S, Singer Y, Srebro N, Cotter A
 "Pegasos: Primal Estimated Sub-Gradient Solver for SVM"
 Mathematical Programming, 2011.

  • Constructor Summary

    Constructors
    Constructor
    Description
    Pegasos(double baseRate, double lambda)
    Constructs a Pegasos optimiser with the specified parameters.

  • Method Summary

    Modifier and Type
    Method
    Description
    Pegasos
    copy()
    Copies a gradient optimiser with it's configuration.
    void
    finalise()
    Finalises the gradient optimisation, setting the parameters to their correct values.
    com.oracle.labs.mlrg.olcut.provenance.ConfiguredObjectProvenance
    getProvenance()
     
    void
    initialise(Parameters parameters)
    Initialises the gradient optimiser.
    void
    reset()
    Resets the optimiser so it's ready to optimise a new Parameters.
    Tensor[]
    step(Tensor[] updates, double weight)
    Take a Tensor array of gradients and transform them according to the current weight and learning rates.
    String
    toString()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface com.oracle.labs.mlrg.olcut.config.Configurable

    postConfig

  • Constructor Details

    • Pegasos

      public Pegasos(double baseRate, double lambda)
      Constructs a Pegasos optimiser with the specified parameters.
      Parameters:
      baseRate - The base learning rate.
      lambda - The regularisation parameter.

  • Method Details