edu.stanford.nlp.maxent.iis
Class LambdaSolve

java.lang.Object
  extended by edu.stanford.nlp.maxent.iis.LambdaSolve
Direct Known Subclasses:
LambdaSolveTagger

public class LambdaSolve
extends Object

This is the main class that does the core computation in IIS.

Author:
Kristina Toutanova

Field Summary
 boolean convertValues
           
 double eps
           
 boolean fixedFnumXY
          This flag is true if all (x,y) have the same f# in which case the newton equation solving is avoided
protected  byte[][] fnumArr
          This contains the number of features active for each pair (x,y)
protected  double[] ftildeArr
          This is an array of empirical expectations for the features
 double[] lambda
          These are the model parameters that have to be learned
 boolean[] lambda_converged
           
 double newtonerr
           
 Problem p
           
protected  double[][] probConds
          Conditional probabilities.
 boolean weightRanks
           
protected  double[] zlambda
          Normalization factors, one for each x.
 
Constructor Summary
LambdaSolve()
           
LambdaSolve(Problem p1, double eps1, double nerr1)
           
LambdaSolve(String filename)
          Reads the lambda parameters from a file.
 
Method Summary
 boolean checkCorrectness()
          Check whether the constraints are satisfied, the probabilities sum to one, etc.
static double divide(double first, double second)
          Given a numerator and denominator in log form, this calculates the conditional model probabilities.
 double expectedValue()
          Each pair x,y has a value in p.data.values[x][y]
protected  double fnum(int x, int y)
           
 double GainCompute(Feature f, double errorGain)
          Computes the gain from a feature.
 double[] getDerivatives()
          assuming we have the lambdas in the array and we need only the derivatives now.
 double[] getDerivativesExpectedValue()
          assuming we have the probConds[x][y] , compute the derivatives for the expectedValue function
 double[] getDerivativesLossDomination()
          Using the arrays calculated when computing the loss, it should not be too hard to get the derivatives.
 double[] getDerivativesNeg()
          assuming we have the lambdas in the array and we need only the derivatives now.
 void improvedIterative()
          Iterate until convergence.
 void improvedIterative(int iters)
          Does a fixed number of IIS iterations.
 double logLikelihood()
           
 double logLikelihoodNeg()
          Calculate the log-likelihood from scratch, hashing the conditional probabilities in pcond, which we will use later.
 double logLikelihoodScratch()
          calculate the log likelihood from scratch, hashing the conditional probabilities in pcond which we will use for the derivative later.
 double lossDomination()
          calculate the loss for Dom ranking using the numbers in p.data.values to determine domination relationships in the graphs if values[x][y]> values[x][y'] then there is an edge (x,y)->(x,y')
static void main(String[] args)
          With arguments, this will print out the lambda parameters of a bunch of .lam files (which are assumed to all be the same size).
 double pcond(int y, int x)
           
 void print()
          Print out p(y|x) for all pairs to the standard output.
static double[] read_lambdas(InDataStreamFile rf)
          Read the lambdas from the stream.
 void readL(String filename)
          Read the lambdas from the file.
static void save_lambdas(OutDataStreamFile rf, double[] lambdas)
          Writes the lambdas to a stream.
 void save_lambdas(String filename)
          Writes the lambda feature weights to the file.
 void setBinary()
           
 void setNonBinary()
           
 void transformValues()
          This is a specialized procedure to change the values of parses for semantic ranking.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

lambda

public double[] lambda
These are the model parameters that have to be learned


lambda_converged

public boolean[] lambda_converged

eps

public double eps

newtonerr

public double newtonerr

fixedFnumXY

public boolean fixedFnumXY
This flag is true if all (x,y) have the same f# in which case the newton equation solving is avoided


p

public Problem p

probConds

protected double[][] probConds
Conditional probabilities.


zlambda

protected double[] zlambda
Normalization factors, one for each x. (CDM questions 2008: Are these only at training time? Can we not allocate it at test time (unlike what LambdaSolveTagger now does)? Is the place where it is set from ySize wrong?


fnumArr

protected byte[][] fnumArr
This contains the number of features active for each pair (x,y)


ftildeArr

protected double[] ftildeArr
This is an array of empirical expectations for the features


weightRanks

public boolean weightRanks

convertValues

public boolean convertValues
Constructor Detail

LambdaSolve

public LambdaSolve(Problem p1,
                   double eps1,
                   double nerr1)

LambdaSolve

public LambdaSolve(String filename)
Reads the lambda parameters from a file.

Parameters:
filename - File to read from

LambdaSolve

public LambdaSolve()
Method Detail

setNonBinary

public void setNonBinary()

setBinary

public void setBinary()

transformValues

public void transformValues()
This is a specialized procedure to change the values of parses for semantic ranking. The highest value is changed to 2/3 and values of 1 are changed to 1/(3*numones). 0 is unchanged this is used to rank higher the ordering for the best parse values are in p.data.values


improvedIterative

public void improvedIterative()
Iterate until convergence. I usually use the other method that does a fixed number of iterations.


improvedIterative

public void improvedIterative(int iters)
Does a fixed number of IIS iterations.

Parameters:
iters - Number of iterations to run

pcond

public double pcond(int y,
                    int x)

fnum

protected double fnum(int x,
                      int y)

checkCorrectness

public boolean checkCorrectness()
Check whether the constraints are satisfied, the probabilities sum to one, etc. Prints out a message if there is something wrong.


GainCompute

public double GainCompute(Feature f,
                          double errorGain)
Computes the gain from a feature. Used for feature selection.


print

public void print()
Print out p(y|x) for all pairs to the standard output.


save_lambdas

public void save_lambdas(String filename)
Writes the lambda feature weights to the file. Can be read later with readL. This method opens a new file and closes it after writing it.

Parameters:
filename - The file to write the weights to.

save_lambdas

public static void save_lambdas(OutDataStreamFile rf,
                                double[] lambdas)
Writes the lambdas to a stream.


readL

public void readL(String filename)
Read the lambdas from the file. The file contains the number of lambda weights (int) followed by the weights. Historical note: The file does not contain xSize and ySize as for the method read(String).

Parameters:
filename - The file to read from

read_lambdas

public static double[] read_lambdas(InDataStreamFile rf)
Read the lambdas from the stream.

Parameters:
rf - Stream to read from.
Returns:
An array of lambda values read from the stream.

logLikelihood

public double logLikelihood()
Returns:
The loglikelihood of the empirical distribution as predicted by the model p.

divide

public static double divide(double first,
                            double second)
Given a numerator and denominator in log form, this calculates the conditional model probabilities.

Returns:
Math.exp(first)/Math.exp(second);

main

public static void main(String[] args)
With arguments, this will print out the lambda parameters of a bunch of .lam files (which are assumed to all be the same size). (Without arguments, it does some creaky old self-test.)

Parameters:
args - command line arguments

logLikelihoodNeg

public double logLikelihoodNeg()
Calculate the log-likelihood from scratch, hashing the conditional probabilities in pcond, which we will use later. This is for a different model, in which all features effectively get negative weights this model is easier to use for heauristic search p(ti|s)=exp(sum_j{-(e^lambda_j)*f_j(ti)})

Returns:
The negative log likelihood of the data

logLikelihoodScratch

public double logLikelihoodScratch()
calculate the log likelihood from scratch, hashing the conditional probabilities in pcond which we will use for the derivative later.

Returns:
The log likelihood of the data

getDerivatives

public double[] getDerivatives()
assuming we have the lambdas in the array and we need only the derivatives now.


getDerivativesNeg

public double[] getDerivativesNeg()
assuming we have the lambdas in the array and we need only the derivatives now. this is for the case where the model is parameterezied such that all weights are negative see also logLikelihoodNeg


expectedValue

public double expectedValue()
Each pair x,y has a value in p.data.values[x][y]

Returns:
- expected value of corpus -sum_xy (ptilde(x,y)*value(x,y)*pcond(x,y))

getDerivativesExpectedValue

public double[] getDerivativesExpectedValue()
assuming we have the probConds[x][y] , compute the derivatives for the expectedValue function

Returns:
The derivatives of the expected

lossDomination

public double lossDomination()
calculate the loss for Dom ranking using the numbers in p.data.values to determine domination relationships in the graphs if values[x][y]> values[x][y'] then there is an edge (x,y)->(x,y')

Returns:
The loss

getDerivativesLossDomination

public double[] getDerivativesLossDomination()
Using the arrays calculated when computing the loss, it should not be too hard to get the derivatives.

Returns:
The derivative of the loss


Stanford NLP Group