edu.stanford.nlp.classify
Class LogConditionalObjectiveFunction<L,F>
java.lang.Object
edu.stanford.nlp.optimization.AbstractCachingDiffFunction
edu.stanford.nlp.optimization.AbstractStochasticCachingDiffFunction
edu.stanford.nlp.optimization.AbstractStochasticCachingDiffUpdateFunction
edu.stanford.nlp.classify.LogConditionalObjectiveFunction<L,F>
- All Implemented Interfaces:
- DiffFunction, Function, HasInitial
- Direct Known Subclasses:
- AdaptedGaussianPriorObjectiveFunction
public class LogConditionalObjectiveFunction<L,F>
- extends AbstractStochasticCachingDiffUpdateFunction
Maximizes the conditional likelihood with a given prior.
- Author:
- Dan Klein, Galen Andrew, Chris Cox (merged w/ SumConditionalObjectiveFunction, 2/16/05), Sarah Spikes (Templatization, allowing an Iterable> to be passed in instead of a GeneralDataset), Angel Chang (support in place SGD - extend AbstractStochasticCachingDiffUpdateFunction)
Fields inherited from class edu.stanford.nlp.optimization.AbstractStochasticCachingDiffFunction |
allIndices, curElement, finiteDifferenceStepSize, gradPerturbed, hasNewVals, HdotV, lastBatch, lastBatchSize, lastElement, lastVBatch, lastXBatch, method, randGenerator, recalculatePrevBatch, returnPreviousValues, sampleMethod, scaleUp, thisBatch, xPerturbed |
Constructor Summary |
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset)
|
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
float[] dataWeights,
LogPrior prior)
|
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
LogPrior prior)
|
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
LogPrior prior,
boolean useSumCondObjFun)
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
double[][] values,
int[] labels,
int intPrior,
double sigma,
double epsilon)
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels)
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
boolean useSumCondObjFun)
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
float[] dataweights)
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
float[] dataweights,
LogPrior prior)
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
int intPrior,
double sigma,
double epsilon)
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
LogPrior prior)
|
LogConditionalObjectiveFunction(Iterable<Datum<L,F>> dataIterable,
LogPrior logPrior,
Index<F> featureIndex,
Index<L> labelIndex)
|
Method Summary |
protected void |
calculate(double[] x)
Calculate the conditional likelihood. |
void |
calculateStochastic(double[] x,
double[] v,
int[] batch)
calculateStochastic needs to calculate a stochastic approximation to the derivative and value of
of a function for a given batch of the data. |
protected void |
calculateStochasticAlgorithmicDifferentiation(double[] x,
double[] v,
int[] batch)
|
void |
calculateStochasticFiniteDifference(double[] x,
double[] v,
double h,
int[] batch)
|
void |
calculateStochasticGradientOnly(double[] x,
int[] batch)
|
double |
calculateStochasticUpdate(double[] x,
double xscale,
int[] batch,
double gain)
Performs stochastic update of weights x (scaled by xscale) based
on samples indexed by batch |
int |
dataDimension()
Data dimension must return the size of the data used by the function. |
int |
domainDimension()
Returns the number of dimensions in the function's domain |
protected int |
indexOf(int f,
int c)
|
protected void |
rvfcalculate(double[] x)
Calculate conditional likelihood for datasets with real-valued features. |
void |
setPrior(LogPrior prior)
|
void |
setUseSumCondObjFun(boolean value)
|
double[][] |
to2D(double[] x)
|
double |
valueAt(double[] x,
double xscale,
int[] batch)
Computes value of function for specified value of x (scaled by xcale)
only over samples indexed by batch |
Methods inherited from class edu.stanford.nlp.optimization.AbstractStochasticCachingDiffFunction |
clearCache, decrementBatch, derivativeAt, derivativeAt, getBatch, HdotVAt, HdotVAt, HdotVAt, incrementBatch, incrementRandom, initial, lastDerivative, lastValue, scaleUp, setValue, valueAt, valueAt |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
prior
protected LogPrior prior
numFeatures
protected int numFeatures
numClasses
protected int numClasses
data
protected int[][] data
dataIterable
protected Iterable<Datum<L,F>> dataIterable
values
protected double[][] values
labels
protected int[] labels
dataweights
protected float[] dataweights
derivativeNumerator
protected double[] derivativeNumerator
xAD
protected DoubleAD[] xAD
priorDerivative
protected double[] priorDerivative
derivativeAD
protected DoubleAD[] derivativeAD
sums
protected DoubleAD[] sums
probs
protected DoubleAD[] probs
labelIndex
protected Index<L> labelIndex
featureIndex
protected Index<F> featureIndex
useIterable
protected boolean useIterable
useSummedConditionalLikelihood
protected boolean useSummedConditionalLikelihood
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
LogPrior prior)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
float[] dataWeights,
LogPrior prior)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
LogPrior prior,
boolean useSumCondObjFun)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(Iterable<Datum<L,F>> dataIterable,
LogPrior logPrior,
Index<F> featureIndex,
Index<L> labelIndex)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
boolean useSumCondObjFun)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
LogPrior prior)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
float[] dataweights)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
float[] dataweights,
LogPrior prior)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
int intPrior,
double sigma,
double epsilon)
LogConditionalObjectiveFunction
public LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
double[][] values,
int[] labels,
int intPrior,
double sigma,
double epsilon)
setPrior
public void setPrior(LogPrior prior)
domainDimension
public int domainDimension()
- Description copied from interface:
Function
- Returns the number of dimensions in the function's domain
- Specified by:
domainDimension
in interface Function
- Specified by:
domainDimension
in class AbstractCachingDiffFunction
- Returns:
- the number of domain dimensions
dataDimension
public int dataDimension()
- Description copied from class:
AbstractStochasticCachingDiffFunction
- Data dimension must return the size of the data used by the function.
- Specified by:
dataDimension
in class AbstractStochasticCachingDiffFunction
indexOf
protected int indexOf(int f,
int c)
to2D
public double[][] to2D(double[] x)
calculate
protected void calculate(double[] x)
- Calculate the conditional likelihood.
If
useSummedConditionalLikelihood
is false
(the default),
this calculates standard(product) CL, otherwise this calculates summed CL.
What's the difference? See Klein and Manning's 2002 EMNLP paper.
- Specified by:
calculate
in class AbstractCachingDiffFunction
calculateStochastic
public void calculateStochastic(double[] x,
double[] v,
int[] batch)
- Description copied from class:
AbstractStochasticCachingDiffFunction
- calculateStochastic needs to calculate a stochastic approximation to the derivative and value of
of a function for a given batch of the data. The approximation to the derivative must be stored
in the array
derivative
, the approximation to the value in value
and the approximation to the Hessian vector product H.v in the array HdotV
. Note
that the hessian vector product is used primarily with the Stochastic Meta Descent optimization
routine SMDMinimizer
.
Important: The stochastic approximation must be such that the sum of all stochastic calculations over
each of the batches in the data must equal the full calculation. i.e. for a data set of size 100
the sum of the gradients for batches 1-10 , 11-20 , 21-30 .... 91-100 must be the same as the gradient
for the full calculation (at the very least in expectation). Be sure to take into account the priors.
- Specified by:
calculateStochastic
in class AbstractStochasticCachingDiffFunction
- Parameters:
x
- - value to evaluate atv
- - the vector for the Hessian vector product H.vbatch
- - an array containing the indices of the data to use in the calculation, this array is being calculated
internal to the abstract, and only needs to be handled not generated by the implemenation.
calculateStochasticFiniteDifference
public void calculateStochasticFiniteDifference(double[] x,
double[] v,
double h,
int[] batch)
calculateStochasticGradientOnly
public void calculateStochasticGradientOnly(double[] x,
int[] batch)
valueAt
public double valueAt(double[] x,
double xscale,
int[] batch)
- Description copied from class:
AbstractStochasticCachingDiffUpdateFunction
- Computes value of function for specified value of x (scaled by xcale)
only over samples indexed by batch
- Specified by:
valueAt
in class AbstractStochasticCachingDiffUpdateFunction
- Parameters:
x
- - unscaled weightsxscale
- - how much to scale x by when performing calculationsbatch
- - indices of which samples to compute function over
- Returns:
- value of function at specified x (scaled by xscale) for samples
calculateStochasticUpdate
public double calculateStochasticUpdate(double[] x,
double xscale,
int[] batch,
double gain)
- Description copied from class:
AbstractStochasticCachingDiffUpdateFunction
- Performs stochastic update of weights x (scaled by xscale) based
on samples indexed by batch
- Specified by:
calculateStochasticUpdate
in class AbstractStochasticCachingDiffUpdateFunction
- Parameters:
x
- - unscaled weightsxscale
- - how much to scale x by when performing calculationsbatch
- - indices of which samples to compute function overgain
- - how much to scale adjustments to x
- Returns:
- value of function at specified x (scaled by xscale) for samples
calculateStochasticAlgorithmicDifferentiation
protected void calculateStochasticAlgorithmicDifferentiation(double[] x,
double[] v,
int[] batch)
rvfcalculate
protected void rvfcalculate(double[] x)
- Calculate conditional likelihood for datasets with real-valued features.
Currently this can calculate CL only (no support for SCL).
TODO: sum-conditional obj. fun. with RVFs.
setUseSumCondObjFun
public void setUseSumCondObjFun(boolean value)
Stanford NLP Group