edu.stanford.nlp.classify
Class PRCurve

java.lang.Object
  extended by edu.stanford.nlp.classify.PRCurve

public class PRCurve
extends java.lang.Object

Author:
Kristina Toutanova May 23, 2005 A class to create recall-precision curves given scores used to fit the best monotonic function for logistic regression and svms

Constructor Summary
PRCurve(java.util.List<Pair<java.lang.Double,java.lang.Integer>> dataScores)
           
PRCurve(java.lang.String filename)
          reads scores with classes from a file, sorts by score and creates the arrays
PRCurve(java.lang.String filename, boolean svm)
          reads scores with classes from a file, sorts by score and creates the arrays
 
Method Summary
 double accuracy()
           
static boolean correct(double score, int cls)
           
 double cwa()
          confidence weighted accuracy assuming the scores are probabilities and using .5 as treshold
 int[] cwaArray()
          confidence weighted accuracy assuming the scores are probabilities and using .5 as treshold
static double f1(int tp, int fp, int fn)
           
 double fmeasure(int recall)
          what is the f-measure at this recall if we look at the score as the probability of class 1 given x as if coming from logistic regression same as logPrecision but calculating f-measure
 double fmeasure(int numleft, int numright)
          the f-measure if we just guess as negativ the first numleft and guess as poitive the last numright
 void init(java.util.List<Pair<java.lang.Double,java.lang.Integer>> dataScores)
           
 void initMC(java.util.ArrayList<Triple<java.lang.Double,java.lang.Integer,java.lang.Integer>> dataScores)
           
 double logLikelihood()
          assuming the scores are probability of 1 given x
 int logPrecision(int recall)
          what is the precision at this recall if we look at the score as the probability of class 1 given x as if coming from logistic regression
static void main(java.lang.String[] args)
           
 double opFmeasure()
           
 double optFmeasure(int recall)
          what is the optimal f-measure we can achieve given recall guesses using the optimal monotonic function
 double optimalAccuracy()
           
 double optimalCwa()
          optimal confidence weighted accuracy assuming for each recall we can fit an optimal monotonic function
 int[] optimalCwaArray()
          confidence weighted accuracy assuming the scores are probabilities and using .5 as treshold
 int precision(int recall)
          what is the best precision at the given recall
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

PRCurve

public PRCurve(java.lang.String filename)
reads scores with classes from a file, sorts by score and creates the arrays


PRCurve

public PRCurve(java.lang.String filename,
               boolean svm)
reads scores with classes from a file, sorts by score and creates the arrays


PRCurve

public PRCurve(java.util.List<Pair<java.lang.Double,java.lang.Integer>> dataScores)
Method Detail

optimalAccuracy

public double optimalAccuracy()

accuracy

public double accuracy()

init

public void init(java.util.List<Pair<java.lang.Double,java.lang.Integer>> dataScores)

initMC

public void initMC(java.util.ArrayList<Triple<java.lang.Double,java.lang.Integer,java.lang.Integer>> dataScores)

precision

public int precision(int recall)
what is the best precision at the given recall


f1

public static double f1(int tp,
                        int fp,
                        int fn)

fmeasure

public double fmeasure(int numleft,
                       int numright)
the f-measure if we just guess as negativ the first numleft and guess as poitive the last numright


logPrecision

public int logPrecision(int recall)
what is the precision at this recall if we look at the score as the probability of class 1 given x as if coming from logistic regression


optFmeasure

public double optFmeasure(int recall)
what is the optimal f-measure we can achieve given recall guesses using the optimal monotonic function


opFmeasure

public double opFmeasure()

fmeasure

public double fmeasure(int recall)
what is the f-measure at this recall if we look at the score as the probability of class 1 given x as if coming from logistic regression same as logPrecision but calculating f-measure

Parameters:
recall - make this many guesses for which we are most confident

logLikelihood

public double logLikelihood()
assuming the scores are probability of 1 given x


cwa

public double cwa()
confidence weighted accuracy assuming the scores are probabilities and using .5 as treshold


cwaArray

public int[] cwaArray()
confidence weighted accuracy assuming the scores are probabilities and using .5 as treshold


optimalCwaArray

public int[] optimalCwaArray()
confidence weighted accuracy assuming the scores are probabilities and using .5 as treshold


optimalCwa

public double optimalCwa()
optimal confidence weighted accuracy assuming for each recall we can fit an optimal monotonic function


correct

public static boolean correct(double score,
                              int cls)

main

public static void main(java.lang.String[] args)


Stanford NLP Group