edu.stanford.nlp.maxent.iis
Class LambdaSolve

java.lang.Object
  extended by edu.stanford.nlp.maxent.iis.LambdaSolve
Direct Known Subclasses:
LambdaSolveTagger

public class LambdaSolve
extends Object

This is the main class that does the core computation in IIS.

Author:
Kristina Toutanova

Field Summary
 boolean convertValues
           
 double eps
           
 boolean fixedFnumXY
          This flag is true if all (x,y) have the same f# in which case the newton equation solving is avoided
 byte[][] fnumArr
          This contains the number of features active for each pair (x,y)
 double[] ftildeArr
          This is an array of empirical expectations for the features
 double[] lambda
          These are the model parameters that have to be learned
 boolean[] lambda_converged
           
 double newtonerr
           
 Problem p
           
 double[][] probConds
          Conditional probabilities.
static boolean smooth
           
 boolean weightRanks
           
 double[] zlambda
          Normalization factors, one for each x
 
Constructor Summary
LambdaSolve()
           
LambdaSolve(Problem p1, double eps1, double nerr1)
           
LambdaSolve(String filename)
          reads the prameters from a file
 
Method Summary
 boolean checkCorrectness()
          Check whether the constraints are satisfied, the probabilities sum to one, etc.
static double divide(double first, double second)
          Given a numerator and denominator in log form, this calculates the conditional model probabilities.
 double expectedValue()
          Each pair x,y has a value in p.data.values[x][y]
protected  double fnum(int x, int y)
           
 double GainCompute(Feature f, double errorGain)
          Computes the gain from a feature.
 double[] getDerivatives()
          assuming we have the lambdas in the array and we need only the derivatives now.
 double[] getDerivativesExpectedValue()
          assuming we have the probConds[x][y] , compute the derivatives for the expectedValue function
 double[] getDerivativesLossDomination()
          using the arrays calculated when computing the loss, it should not be too hard to get the derivatives
 double[] getDerivativesNeg()
          assuming we have the lambdas in the array and we need only the derivatives now.
 void ImprovedIterative()
          Iterate until convergence.
 void ImprovedIterative(int iters)
          Does a fixed number of IIS iterations.
 double logLikelihood()
           
 double logLikelihoodNeg()
          Calculate the log-likelihood from scratch, hashing the conditional probabilities in pcond, which we will use later.
 double logLikelihoodScratch()
          calculate the log likelihood from scratch, hashing the conditional probabilities in pcond which we will use for the derivative later.
 double lossDomination()
          calculate the loss for Dom ranking using the numbers in p.data.values to determine domination relationships in the graphs if values[x][y]> values[x][y'] then there is an edge (x,y)->(x,y')
static void main(String[] args)
          With arguments, this will print out the lambda parameters of a bunch of .lam files (which are assumed to all be the same size).
 double pcond(int y, int x)
           
 void print()
          Print out p(y|x) for all pairs to the standard output.
static double[] read_lambdas(InDataStreamFile rf)
           
 void read(String filename)
          Read the model parameters from a file ( only the lambdas )
protected  void readL(InDataStreamFile rf)
          Read the lambdas from the stream.
 void readL(String filename)
          Read the lambdas from the file.
static void save_lambdas(OutDataStreamFile rf, double[] lambdas)
          Writes the lambdas to a stream
 void save_lambdas(String filename)
          Writes the lambdas to the file.
 void save(OutDataStreamFile file)
          Saves xSize,ySize and the lambdas to the file.
 void save(String filename)
          Saves xSize,ySize and the lambdas to the file.
 void setBinary()
           
 void setNonBinary()
           
 void transformValues()
          This is a specialized procedure to change the values of parses for semantic ranking.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

lambda

public double[] lambda
These are the model parameters that have to be learned


lambda_converged

public boolean[] lambda_converged

eps

public double eps

newtonerr

public double newtonerr

fixedFnumXY

public boolean fixedFnumXY
This flag is true if all (x,y) have the same f# in which case the newton equation solving is avoided


p

public Problem p

probConds

public double[][] probConds
Conditional probabilities.


zlambda

public double[] zlambda
Normalization factors, one for each x


fnumArr

public byte[][] fnumArr
This contains the number of features active for each pair (x,y)


ftildeArr

public double[] ftildeArr
This is an array of empirical expectations for the features


smooth

public static boolean smooth

weightRanks

public boolean weightRanks

convertValues

public boolean convertValues
Constructor Detail

LambdaSolve

public LambdaSolve(Problem p1,
                   double eps1,
                   double nerr1)

LambdaSolve

public LambdaSolve(String filename)
reads the prameters from a file


LambdaSolve

public LambdaSolve()
Method Detail

setNonBinary

public void setNonBinary()

setBinary

public void setBinary()

transformValues

public void transformValues()
This is a specialized procedure to change the values of parses for semantic ranking. The highest value is changed to 2/3 and values of 1 are changed to 1/(3*numones). 0 is unchanged this is used to rank higher the ordering for the best parse values are in p.data.values


ImprovedIterative

public void ImprovedIterative()
Iterate until convergence. I usually use the other method that does a fixed number of iterations.


ImprovedIterative

public void ImprovedIterative(int iters)
Does a fixed number of IIS iterations.


pcond

public double pcond(int y,
                    int x)

fnum

protected double fnum(int x,
                      int y)

checkCorrectness

public boolean checkCorrectness()
Check whether the constraints are satisfied, the probabilities sum to one, etc. Prints out a message if there is something wrong.


GainCompute

public double GainCompute(Feature f,
                          double errorGain)
Computes the gain from a feature. Used for feature selection.


print

public void print()
Print out p(y|x) for all pairs to the standard output.


save

public void save(String filename)
Saves xSize,ySize and the lambdas to the file. Can be read later with read(String).


save

public void save(OutDataStreamFile file)
Saves xSize,ySize and the lambdas to the file. Can be read later with read(String).


save_lambdas

public void save_lambdas(String filename)
Writes the lambdas to the file. Can be read later with readL.


save_lambdas

public static void save_lambdas(OutDataStreamFile rf,
                                double[] lambdas)
Writes the lambdas to a stream


readL

public void readL(String filename)
Read the lambdas from the file. The file does not have to contain xSize and ySize as for the method read(String).


readL

protected void readL(InDataStreamFile rf)
Read the lambdas from the stream.


read

public void read(String filename)
Read the model parameters from a file ( only the lambdas )


read_lambdas

public static double[] read_lambdas(InDataStreamFile rf)

logLikelihood

public double logLikelihood()
Returns:
The loglikelihood of the empirical distribution as predicted by the model p.

divide

public static double divide(double first,
                            double second)
Given a numerator and denominator in log form, this calculates the conditional model probabilities.

Returns:
Math.exp(first)/Math.exp(second);

main

public static void main(String[] args)
With arguments, this will print out the lambda parameters of a bunch of .lam files (which are assumed to all be the same size). If the first argument is "-pad" then it will read the 2nd argument as a .lam file and write to the third argument a .prob file (the .lam file prepadded with 2 integers -- set to 0). If the first argument is "-strip" then it will read the 2nd argument as a .prob file and write to the third argument a .lam file (the .prob file minus the first two integers).

Parameters:
args - command line arguments

logLikelihoodNeg

public double logLikelihoodNeg()
Calculate the log-likelihood from scratch, hashing the conditional probabilities in pcond, which we will use later. This is for a different model, in which all features effectively get negative weights this model is easier to use for heauristic search p(ti|s)=exp(sum_j{-(e^lambda_j)*f_j(ti)})

Returns:
The negative log likelihood of the data

logLikelihoodScratch

public double logLikelihoodScratch()
calculate the log likelihood from scratch, hashing the conditional probabilities in pcond which we will use for the derivative later.


getDerivatives

public double[] getDerivatives()
assuming we have the lambdas in the array and we need only the derivatives now.


getDerivativesNeg

public double[] getDerivativesNeg()
assuming we have the lambdas in the array and we need only the derivatives now. this is for the case where the model is parameterezied such that all weights are negative see also logLikelihoodNeg


expectedValue

public double expectedValue()
Each pair x,y has a value in p.data.values[x][y]

Returns:
- expected value of corpus -sum_xy (ptilde(x,y)*value(x,y)*pcond(x,y))

getDerivativesExpectedValue

public double[] getDerivativesExpectedValue()
assuming we have the probConds[x][y] , compute the derivatives for the expectedValue function


lossDomination

public double lossDomination()
calculate the loss for Dom ranking using the numbers in p.data.values to determine domination relationships in the graphs if values[x][y]> values[x][y'] then there is an edge (x,y)->(x,y')


getDerivativesLossDomination

public double[] getDerivativesLossDomination()
using the arrays calculated when computing the loss, it should not be too hard to get the derivatives



Stanford NLP Group