edu.stanford.nlp.optimization

Class SGDToQNMinimizer

java.lang.Object

edu.stanford.nlp.optimization.SGDToQNMinimizer

All Implemented Interfaces:

Minimizer<DiffFunction>, java.io.Serializable

public class SGDToQNMinimizer
extends java.lang.Object
implements Minimizer<DiffFunction>, java.io.Serializable

Stochastic Gradient Descent To Quasi Newton Minimizer An experimental minimizer which takes a stochastic function (one implementing AbstractStochasticCachingDiffFunction) and executes SGD for the first couple passes. During the final iterations a series of approximate hessian vector products are built up. These are then passed to the QNminimizer so that it can start right up without the typical delay. Note [2012] The basic idea here is good, but the original ScaledSGDMinimizer wasn't efficient, and so this would be much more useful if rewritten to use the good StochasticInPlaceMinimizer instead.

Since:

1.0

Version:

1.0

Author:

Alex Kleeman

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
double	gain
boolean	outputIterationsToFile
int	QNPasses
int	SGDPasses

Constructor Summary

Constructors

Constructor and Description
SGDToQNMinimizer(double SGDGain, int batchSize, int SGDPasses, int QNPasses)
SGDToQNMinimizer(double SGDGain, int batchSize, int sgdPasses, int qnPasses, int hessSamples, int QNMem)
SGDToQNMinimizer(double SGDGain, int batchSize, int sgdPasses, int qnPasses, int hessSamples, int QNMem, boolean outputToFile)

Method Summary

Modifier and Type	Method and Description
protected java.lang.String	getName()
double[]	minimize(DiffFunction function, double functionTolerance, double[] initial) Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance).
double[]	minimize(DiffFunction function, double functionTolerance, double[] initial, int maxIterations) Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance), but running only for at most maxIterations iterations.
void	shutUp()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

outputIterationsToFile

public boolean outputIterationsToFile

gain

public double gain

SGDPasses

public int SGDPasses

QNPasses

public int QNPasses

Constructor Detail

SGDToQNMinimizer

public SGDToQNMinimizer(double SGDGain,
                        int batchSize,
                        int SGDPasses,
                        int QNPasses)

SGDToQNMinimizer

public SGDToQNMinimizer(double SGDGain,
                        int batchSize,
                        int sgdPasses,
                        int qnPasses,
                        int hessSamples,
                        int QNMem)

SGDToQNMinimizer

public SGDToQNMinimizer(double SGDGain,
                        int batchSize,
                        int sgdPasses,
                        int qnPasses,
                        int hessSamples,
                        int QNMem,
                        boolean outputToFile)

Method Detail

shutUp

public void shutUp()

getName

protected java.lang.String getName()

minimize

public double[] minimize(DiffFunction function,
                         double functionTolerance,
                         double[] initial)

Description copied from interface: Minimizer

Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance).

Specified by:

minimize in interface Minimizer<DiffFunction>

Parameters:

function - The objective function

functionTolerance - A double value

initial - An initial feasible point

Returns:

Unconstrained minimum of function

minimize

public double[] minimize(DiffFunction function,
                         double functionTolerance,
                         double[] initial,
                         int maxIterations)

Description copied from interface: Minimizer

Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance), but running only for at most maxIterations iterations.

Specified by:

minimize in interface Minimizer<DiffFunction>

Parameters:

function - The objective function

functionTolerance - A double value

initial - An initial feasible point

maxIterations - Maximum number of iterations

Returns:

Unconstrained minimum of function