edu.stanford.nlp.optimization
Class QNMinimizer

java.lang.Object
  extended by edu.stanford.nlp.optimization.QNMinimizer
All Implemented Interfaces:
Minimizer<DiffFunction>

public class QNMinimizer
extends Object
implements Minimizer<DiffFunction>

An implementation of L-BFGS for Quasi Newton unconstrained minimization. The general outline of the algorithm is taken from:

Numerical Optimization (second edition) 2006 Jorge Nocedal and Stephen J. Wright
A variety of different options are available.

LINESEARCHES

BACKTRACKING: This routine simply starts with a guess for step size of 1. If the step size doesn't supply a sufficient decrease in the function value the step is updated through step = 0.1*step. This method is certainly simpler, but doesn't allow for an increase in step size, and isn't well suited for Quasi Newton methods. MINPACK: This routine is based off of the implementation used in MINPACK. This routine finds a point satisfying the Wolfe conditions, which state that a point must have a sufficiently smaller function value, and a gradient of smaller magnitude. This provides enough to prove theoretically quadratic convergence. In order to find such a point the linesearch first finds an interval which must contain a satisfying point, and then progressively reduces that interval all using cubic or quadratic interpolation. SCALING: L-BFGS allows the initial guess at the hessian to be updated at each step. Standard BFGS does this by approximating the hessian as a scaled identity matrix. To use this method set the scaleOpt to SCALAR. A better way of approximate the hessian is by using a scaling diagonal matrix. The diagonal can then be updated as more information comes in. This method can be used by setting scaleOpt to DIAGONAL. CONVERGENCE: Previously convergence was gauged by looking at the average decrease per step dividing that by the current value and terminating when that value because smaller than TOL. This method fails when the function value approaches zero, so two other convergence criteria are used. The first stores the initial gradient norm |g0|, then terminates when the new gradient norm, |g| is sufficently smaller: i.e., |g| < eps*|g0| the second checks if |g| < eps*max( 1 , |x| ) which is essentially checking to see if the gradient is numerically zero. Each of these convergence criteria can be turned on or off by setting the flags:
private boolean useAveImprovement = true; private boolean useRelativeNorm = true; private boolean useNumericalZero = true;
To use the QNMinimizer first construct it using
QNMinimizer qn = new QNMinimizer(mem, true)
mem - the number of previous estimate vector pairs to store, generally 15 is plenty. true - this tells the QN to use the MINPACK linesearch with DIAGONAL scaling. false would lead to the use of the criteria used in the old QNMinimizer class. Then call:
qn.minimize(dfunction,convergenceTolerance,initialGuess,maxFunctionEvaluations);

Author:
akleeman

Nested Class Summary
static class QNMinimizer.eLineSearch
           
static class QNMinimizer.eScaling
           
static class QNMinimizer.eState
           
 class QNMinimizer.QNInfo
          The QNInfo class is used to store information about the Quasi Newton update.
 class QNMinimizer.Record
          The Record class is used to collect information about the function value over a series of iterations.
static class QNMinimizer.SurpriseConvergence
           
 
Field Summary
 boolean bracketed
           
 double completionTime
           
 int fevals
           
 int its
           
 boolean outputToFile
           
 boolean success
           
 
Constructor Summary
QNMinimizer()
           
QNMinimizer(FloatFunction monitor)
           
QNMinimizer(Function monitor)
           
QNMinimizer(Function monitor, int m)
           
QNMinimizer(Function monitor, int m, boolean useRobustOptions)
           
QNMinimizer(int m)
           
QNMinimizer(int m, boolean useRobustOptions)
           
 
Method Summary
 float[] minimize(DiffFloatFunction function, float functionTolerance, float[] initial)
           
 double[] minimize(DiffFunction function, double functionTolerance, double[] initial)
          Attempts to find an unconstrained minimum of the objective function starting at initial, within functionTolerance.
 double[] minimize(DiffFunction dfunction, double functionTolerance, double[] initial, int maxFunctionEvaluations)
           
 double[] minimize(DiffFunction dfunction, double functionTolerance, double[] initial, int maxFunctionEvaluations, QNMinimizer.QNInfo qn)
           
 void setHistory(List<double[]> s, List<double[]> y)
           
 void setM(int m)
           
 void setOldOptions()
           
 void setRobustOptions()
           
 void shutUp()
           
 void terminateOnAverageImprovement(boolean toTerminate)
           
 void terminateOnNumericalZero(boolean toTerminate)
           
 void terminateOnRelativeNorm(boolean toTerminate)
           
 void useBacktracking()
           
 void useDiagonalScaling()
           
 void useMinPackSearch()
           
 void useScalarScaling()
           
 boolean wasSuccessful()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

fevals

public int fevals

its

public int its

completionTime

public double completionTime

outputToFile

public boolean outputToFile

success

public boolean success

bracketed

public boolean bracketed
Constructor Detail

QNMinimizer

public QNMinimizer(int m)

QNMinimizer

public QNMinimizer(int m,
                   boolean useRobustOptions)

QNMinimizer

public QNMinimizer()

QNMinimizer

public QNMinimizer(Function monitor)

QNMinimizer

public QNMinimizer(FloatFunction monitor)

QNMinimizer

public QNMinimizer(Function monitor,
                   int m)

QNMinimizer

public QNMinimizer(Function monitor,
                   int m,
                   boolean useRobustOptions)
Method Detail

setOldOptions

public void setOldOptions()

setRobustOptions

public final void setRobustOptions()

terminateOnRelativeNorm

public void terminateOnRelativeNorm(boolean toTerminate)

terminateOnNumericalZero

public void terminateOnNumericalZero(boolean toTerminate)

terminateOnAverageImprovement

public void terminateOnAverageImprovement(boolean toTerminate)

useMinPackSearch

public void useMinPackSearch()

useBacktracking

public void useBacktracking()

useDiagonalScaling

public void useDiagonalScaling()

useScalarScaling

public void useScalarScaling()

wasSuccessful

public boolean wasSuccessful()

shutUp

public void shutUp()

setM

public void setM(int m)

setHistory

public void setHistory(List<double[]> s,
                       List<double[]> y)

minimize

public float[] minimize(DiffFloatFunction function,
                        float functionTolerance,
                        float[] initial)

minimize

public double[] minimize(DiffFunction function,
                         double functionTolerance,
                         double[] initial)
Description copied from interface: Minimizer
Attempts to find an unconstrained minimum of the objective function starting at initial, within functionTolerance.

Specified by:
minimize in interface Minimizer<DiffFunction>
Parameters:
function - the objective function
functionTolerance - a double value
initial - a initial feasible point
Returns:
Unconstrained minimum of function

minimize

public double[] minimize(DiffFunction dfunction,
                         double functionTolerance,
                         double[] initial,
                         int maxFunctionEvaluations)
Specified by:
minimize in interface Minimizer<DiffFunction>

minimize

public double[] minimize(DiffFunction dfunction,
                         double functionTolerance,
                         double[] initial,
                         int maxFunctionEvaluations,
                         QNMinimizer.QNInfo qn)


Stanford NLP Group