Package edu.stanford.nlp.classify

Class LogConditionalObjectiveFunction<L,​F>

  • All Implemented Interfaces:
    DiffFunction, Function, HasInitial
    Direct Known Subclasses:
    AdaptedGaussianPriorObjectiveFunction
    public class LogConditionalObjectiveFunction<L,​F>
extends AbstractStochasticCachingDiffUpdateFunction
    Maximizes the conditional likelihood with a given prior.
    Author:
    Dan Klein, Galen Andrew, Chris Cox (merged w/ SumConditionalObjectiveFunction, 2/16/05), Sarah Spikes (Templatization, allowing an Iterable<Datum<L, F>> to be passed in instead of a GeneralDataset<L, F>), Angel Chang (support in place SGD - extend AbstractStochasticCachingDiffUpdateFunction), Christopher Manning (cleaned out the cruft and sped it up in 2014), Keenon Werling added some multithreading to the batch evaluations

    • Field Detail

      • numFeatures

        protected final int numFeatures

      • numClasses

        protected final int numClasses

      • data

        protected final int[][] data
        Normally, this contains the data. The first index is the datum number, and then there is an array of feature indices for each datum.

      • dataIterable

        protected final Iterable<Datum<L,​F>> dataIterable
        Alternatively, the data may be available from an Iterable in not yet indexed form. (In 2014, it's not clear any code actually uses this option.) And then you need an index for both.

      • labelIndex

        protected final Index<L> labelIndex

      • featureIndex

        protected final Index<F> featureIndex

      • values

        protected final double[][] values
        Same size as data if the features have values; null if the features are binary.

      • labels

        protected final int[] labels
        The label of each data index.

      • dataWeights

        protected final float[] dataWeights

      • useSummedConditionalLikelihood

        protected final boolean useSummedConditionalLikelihood

      • derivativeNumerator

        protected double[] derivativeNumerator
        This is used to cache the numerator in batch methods.

      • priorDerivative

        protected double[] priorDerivative
        The only reason this is around is because the Prior Functions don't handle stochastic calculations yet.

      • parallelGradientCalculation

        protected boolean parallelGradientCalculation
        The flag to tell the gradient computations to multithread over the data. keenon (june 2015): On my machine,

      • threads

        protected int threads
        Multithreading gradient calculations is a bit cheaper if you reuse the threads.

    • Constructor Detail

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(GeneralDataset<L,​F> dataset)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(GeneralDataset<L,​F> dataset,
                                       LogPrior prior)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(GeneralDataset<L,​F> dataset,
                                       float[] dataWeights,
                                       LogPrior prior)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(GeneralDataset<L,​F> dataset,
                                       LogPrior prior,
                                       boolean useSumCondObjFun)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(GeneralDataset<L,​F> dataset,
                                       LogPrior prior,
                                       boolean useSumCondObjFun,
                                       float[] dataWeights)
        Version passing in a GeneralDataset, which may be binary or real-valued features.

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(Iterable<Datum<L,​F>> dataIterable,
                                       LogPrior logPrior,
                                       Index<F> featureIndex,
                                       Index<L> labelIndex)
        Version where an Iterable is passed in for the data. Doesn't support dataWeights.

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(int numFeatures,
                                       int numClasses,
                                       int[][] data,
                                       int[] labels,
                                       boolean useSumCondObjFun)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(int numFeatures,
                                       int numClasses,
                                       int[][] data,
                                       int[] labels)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(int numFeatures,
                                       int numClasses,
                                       int[][] data,
                                       int[] labels,
                                       LogPrior prior)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(int numFeatures,
                                       int numClasses,
                                       int[][] data,
                                       int[] labels,
                                       float[] dataWeights)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(int numFeatures,
                                       int numClasses,
                                       int[][] data,
                                       int[] labels,
                                       float[] dataWeights,
                                       LogPrior prior)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(int numFeatures,
                                       int numClasses,
                                       int[][] data,
                                       int[] labels,
                                       float[] dataWeights,
                                       LogPrior prior,
                                       boolean useSummedConditionalLikelihood)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(int numFeatures,
                                       int numClasses,
                                       int[][] data,
                                       int[] labels,
                                       int intPrior,
                                       double sigma,
                                       double epsilon)

      • LogConditionalObjectiveFunction

        public LogConditionalObjectiveFunction​(int numFeatures,
                                       int numClasses,
                                       int[][] data,
                                       double[][] values,
                                       int[] labels,
                                       int intPrior,
                                       double sigma,
                                       double epsilon)
        For real-valued features. Passing in processed data set.

    • Method Detail

      • domainDimension

        public int domainDimension()
        Description copied from interface: Function
        Returns the number of dimensions in the function's domain
        Returns:
        the number of domain dimensions

      • indexOf

        protected int indexOf​(int f,
                      int c)
        Converts a Phi feature number and class index into an f(x,y) feature index.

      • to2D

        public double[][] to2D​(double[] x)

      • calculate

        protected void calculate​(double[] x)
        Calculate the conditional likelihood. If useSummedConditionalLikelihood is false (the default), this calculates standard(product) CL, otherwise this calculates summed CL. What's the difference? See Klein and Manning's 2002 EMNLP paper.
        Specified by:
        calculate in class AbstractCachingDiffFunction
        Parameters:
        x - The point at which to calculate the function

      • calculateStochastic

        public void calculateStochastic​(double[] x,
                                double[] v,
                                int[] batch)
        This function is used to come up with an estimate of the value / gradient based on only a small portion of the data (referred to as the batchSize for lack of a better term. In this case batch does not mean All!! It should be thought of in the sense of "a small batch of the data".
        Specified by:
        calculateStochastic in class AbstractStochasticCachingDiffFunction
        Parameters:
        x - Value to evaluate at
        v - The vector for the Hessian vector product H.v
        batch - An array containing the indices of the data to use in the calculation, this array is being calculated internal to the abstract, and only needs to be handled not generated by the implementation.

      • calculateStochasticFiniteDifference

        public void calculateStochasticFiniteDifference​(double[] x,
                                                double[] v,
                                                double h,
                                                int[] batch)

      • calculateStochasticGradientLocal

        public void calculateStochasticGradientLocal​(double[] x,
                                             int[] batch)

      • valueAt

        public double valueAt​(double[] x,
                      double xscale,
                      int[] batch)
        Description copied from class: AbstractStochasticCachingDiffUpdateFunction
        Computes value of function for specified value of x (scaled by xScale) only over samples indexed by batch.
        Specified by:
        valueAt in class AbstractStochasticCachingDiffUpdateFunction
        Parameters:
        x - unscaled weights
        xscale - how much to scale x by when performing calculations
        batch - indices of which samples to compute function over
        Returns:
        value of function at specified x (scaled by xScale) for samples

      • calculateStochasticUpdate

        public double calculateStochasticUpdate​(double[] x,
                                        double xscale,
                                        int[] batch,
                                        double gain)
        Description copied from class: AbstractStochasticCachingDiffUpdateFunction
        Performs stochastic update of weights x (scaled by xScale) based on samples indexed by batch.
        Specified by:
        calculateStochasticUpdate in class AbstractStochasticCachingDiffUpdateFunction
        Parameters:
        x - unscaled weights
        xscale - how much to scale x by when performing calculations
        batch - indices of which samples to compute function over
        gain - how much to scale adjustments to x
        Returns:
        value of function at specified x (scaled by xScale) for samples

      • calculateStochasticGradient

        public void calculateStochasticGradient​(double[] x,
                                        int[] batch)
        Description copied from class: AbstractStochasticCachingDiffUpdateFunction
        Performs stochastic gradient calculation based on samples indexed by batch and does not apply regularization. Does not update the parameter values. Typically stores derivative information for later access.
        Specified by:
        calculateStochasticGradient in class AbstractStochasticCachingDiffUpdateFunction
        Parameters:
        x - Unscaled weights
        batch - Indices of which samples to compute function over

      • calculateStochasticAlgorithmicDifferentiation

        protected void calculateStochasticAlgorithmicDifferentiation​(double[] x,
                                                             double[] v,
                                                             int[] batch)

      • rvfcalculate

        protected void rvfcalculate​(double[] x)
        Calculate conditional likelihood for datasets with real-valued features. Currently this can calculate CL only (no support for SCL). TODO: sum-conditional obj. fun. with RVFs.