public class LogConditionalObjectiveFunction<L,F> extends AbstractStochasticCachingDiffUpdateFunction
Iterable<Datum<L, F>>
to be passed in instead of a GeneralDataset<L, F>
), Angel Chang (support in place SGD - extend AbstractStochasticCachingDiffUpdateFunction), Christopher Manning (cleaned out the cruft and sped it up in 2014), Keenon Werling added some multithreading to the batch evaluationsAbstractStochasticCachingDiffFunction.SamplingMethod
Modifier and Type | Field and Description |
---|---|
protected int[][] |
data
Normally, this contains the data.
|
protected java.lang.Iterable<Datum<L,F>> |
dataIterable
Alternatively, the data may be available from an Iterable in not yet
indexed form.
|
protected float[] |
dataWeights |
protected double[] |
derivativeNumerator
This is used to cache the numerator in batch methods.
|
protected Index<F> |
featureIndex |
protected Index<L> |
labelIndex |
protected int[] |
labels
The label of each data index.
|
protected int |
numClasses |
protected int |
numFeatures |
protected boolean |
parallelGradientCalculation
The flag to tell the gradient computations to multithread over the data.
|
protected LogPrior |
prior |
protected double[] |
priorDerivative
The only reason this is around is because the Prior Functions don't handle stochastic calculations yet.
|
protected int |
threads
Multithreading gradient calculations is a bit cheaper if you reuse the threads.
|
protected boolean |
useSummedConditionalLikelihood |
protected double[][] |
values
Same size as data if the features have values; null if the features are binary.
|
skipValCalc
allIndices, curElement, finiteDifferenceStepSize, gradPerturbed, hasNewVals, HdotV, lastBatch, lastBatchSize, lastElement, lastVBatch, lastXBatch, method, randGenerator, recalculatePrevBatch, returnPreviousValues, sampleMethod, scaleUp, thisBatch, xPerturbed
derivative, value
Constructor and Description |
---|
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset) |
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
float[] dataWeights,
LogPrior prior) |
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
LogPrior prior) |
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
LogPrior prior,
boolean useSumCondObjFun) |
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
LogPrior prior,
boolean useSumCondObjFun,
float[] dataWeights)
Version passing in a GeneralDataset, which may be binary or real-valued features.
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
double[][] values,
int[] labels,
int intPrior,
double sigma,
double epsilon)
For real-valued features.
|
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels) |
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
boolean useSumCondObjFun) |
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
float[] dataWeights) |
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
float[] dataWeights,
LogPrior prior) |
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
float[] dataWeights,
LogPrior prior,
boolean useSummedConditionalLikelihood) |
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
int intPrior,
double sigma,
double epsilon) |
LogConditionalObjectiveFunction(int numFeatures,
int numClasses,
int[][] data,
int[] labels,
LogPrior prior) |
LogConditionalObjectiveFunction(java.lang.Iterable<Datum<L,F>> dataIterable,
LogPrior logPrior,
Index<F> featureIndex,
Index<L> labelIndex)
Version where an Iterable is passed in for the data.
|
Modifier and Type | Method and Description |
---|---|
protected void |
calculate(double[] x)
Calculate the conditional likelihood.
|
void |
calculateStochastic(double[] x,
double[] v,
int[] batch)
This function is used to come up with an estimate of the value / gradient based on only a small
portion of the data (referred to as the batchSize for lack of a better term.
|
protected void |
calculateStochasticAlgorithmicDifferentiation(double[] x,
double[] v,
int[] batch) |
void |
calculateStochasticFiniteDifference(double[] x,
double[] v,
double h,
int[] batch) |
void |
calculateStochasticGradient(double[] x,
int[] batch)
Performs stochastic gradient calculation based
on samples indexed by batch and does not apply regularization.
|
void |
calculateStochasticGradientLocal(double[] x,
int[] batch) |
double |
calculateStochasticUpdate(double[] x,
double xscale,
int[] batch,
double gain)
Performs stochastic update of weights x (scaled by xScale) based
on samples indexed by batch.
|
int |
dataDimension()
Data dimension must return the size of the data used by the function.
|
int |
domainDimension()
Returns the number of dimensions in the function's domain
|
protected int |
indexOf(int f,
int c)
Converts a Phi feature number and class index into an f(x,y) feature index.
|
protected void |
rvfcalculate(double[] x)
Calculate conditional likelihood for datasets with real-valued features.
|
double[][] |
to2D(double[] x) |
double |
valueAt(double[] x,
double xscale,
int[] batch)
Computes value of function for specified value of x (scaled by xScale)
only over samples indexed by batch.
|
calculateStochasticGradient, calculateStochasticUpdate, getSample, valueAt
clearCache, decrementBatch, derivativeAt, derivativeAt, getBatch, HdotVAt, HdotVAt, HdotVAt, incrementBatch, incrementRandom, initial, lastDerivative, lastValue, scaleUp, valueAt, valueAt
copy, derivativeAt, ensure, getDerivative, gradientCheck, gradientCheck, randomInitial, valueAt
protected final LogPrior prior
protected final int numFeatures
protected final int numClasses
protected final int[][] data
protected final java.lang.Iterable<Datum<L,F>> dataIterable
protected final double[][] values
protected final int[] labels
protected final float[] dataWeights
protected final boolean useSummedConditionalLikelihood
protected double[] derivativeNumerator
protected double[] priorDerivative
protected boolean parallelGradientCalculation
protected int threads
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset)
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset, LogPrior prior)
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset, float[] dataWeights, LogPrior prior)
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset, LogPrior prior, boolean useSumCondObjFun)
public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset, LogPrior prior, boolean useSumCondObjFun, float[] dataWeights)
public LogConditionalObjectiveFunction(java.lang.Iterable<Datum<L,F>> dataIterable, LogPrior logPrior, Index<F> featureIndex, Index<L> labelIndex)
public LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, boolean useSumCondObjFun)
public LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels)
public LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, LogPrior prior)
public LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, float[] dataWeights)
public LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, float[] dataWeights, LogPrior prior)
public LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, float[] dataWeights, LogPrior prior, boolean useSummedConditionalLikelihood)
public LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, int intPrior, double sigma, double epsilon)
public LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, double[][] values, int[] labels, int intPrior, double sigma, double epsilon)
public int domainDimension()
Function
public int dataDimension()
AbstractStochasticCachingDiffFunction
dataDimension
in class AbstractStochasticCachingDiffFunction
protected int indexOf(int f, int c)
public double[][] to2D(double[] x)
protected void calculate(double[] x)
useSummedConditionalLikelihood
is false
(the default),
this calculates standard(product) CL, otherwise this calculates summed CL.
What's the difference? See Klein and Manning's 2002 EMNLP paper.calculate
in class AbstractCachingDiffFunction
x
- The point at which to calculate the functionpublic void calculateStochastic(double[] x, double[] v, int[] batch)
calculateStochastic
in class AbstractStochasticCachingDiffFunction
x
- Value to evaluate atv
- The vector for the Hessian vector product H.vbatch
- An array containing the indices of the data to use in the calculation, this array is being calculated
internal to the abstract, and only needs to be handled not generated by the implementation.public void calculateStochasticFiniteDifference(double[] x, double[] v, double h, int[] batch)
public void calculateStochasticGradientLocal(double[] x, int[] batch)
public double valueAt(double[] x, double xscale, int[] batch)
AbstractStochasticCachingDiffUpdateFunction
valueAt
in class AbstractStochasticCachingDiffUpdateFunction
x
- unscaled weightsxscale
- how much to scale x by when performing calculationsbatch
- indices of which samples to compute function overpublic double calculateStochasticUpdate(double[] x, double xscale, int[] batch, double gain)
AbstractStochasticCachingDiffUpdateFunction
calculateStochasticUpdate
in class AbstractStochasticCachingDiffUpdateFunction
x
- unscaled weightsxscale
- how much to scale x by when performing calculationsbatch
- indices of which samples to compute function overgain
- how much to scale adjustments to xpublic void calculateStochasticGradient(double[] x, int[] batch)
AbstractStochasticCachingDiffUpdateFunction
calculateStochasticGradient
in class AbstractStochasticCachingDiffUpdateFunction
x
- Unscaled weightsbatch
- Indices of which samples to compute function overprotected void calculateStochasticAlgorithmicDifferentiation(double[] x, double[] v, int[] batch)
protected void rvfcalculate(double[] x)