edu.stanford.nlp.tagger.maxent
Class TestSentence

java.lang.Object
  extended by edu.stanford.nlp.tagger.maxent.TestSentence
All Implemented Interfaces:
SequenceModel

public class TestSentence
extends Object
implements SequenceModel

Author:
Kristina Toutanova, Michel Galley

Field Summary
protected  String[] correctTags
           
protected static boolean DBG
           
protected static boolean doDeterministicTagExpansion
           
protected  String encoding
           
protected  String[] finalTags
           
protected static int kBestSize
           
protected  double[][] localContextScores
           
protected  Map<String,double[]> localScores
           
protected  MaxentTagger maxentTagger
           
protected static String naTag
           
protected  List<HasWord> origWords
           
protected  PairsHolder pairs
           
protected  List<String> sent
           
protected  int size
           
protected  String tagSeparator
           
protected static boolean VERBOSE
           
 
Constructor Summary
TestSentence(MaxentTagger maxentTagger)
           
TestSentence(MaxentTagger maxentTagger, String[] s, String[] correctTags, PrintFile pf, Dictionary wrongWords, boolean verboseResults)
           
 
Method Summary
protected  String[] append(String[] tags, String word)
           
protected  void calculateProbs(double[][][] probabilities)
          calculateProbs puts log probs of taggings in the probabilities array.
protected  void cleanUpScorer()
          clean-up after the scorer
protected  double[] getHistories(String[] tags, History h)
           
 int[] getPossibleValues(int pos)
          0...leftWindow-1 etc are null, leftWindow...length+leftWindow-1 are words, length+leftWindow...length+leftWindow+rightWindow-1 are null;
protected  void init()
           
protected  void initializeScorer()
           
 int leftWindow()
          How many positions to the left a position is dependent on.
 int length()
           
protected  void revert(int prevSize)
           
 int rightWindow()
          How many positions to the right a position is dependent on.
 double scoreOf(int[] sequence)
          Computes the score assigned by this model to the whole sequence.
 double scoreOf(int[] tags, int pos)
          Computes the unnormalized log conditional distribution over values of the element at position pos in the sequence, conditioned on the values of the elements in all other positions of the provided sequence.
 double[] scoresOf(int[] tags, int pos)
          Computes the unnormalized log conditional distribution over values of the element at position pos in the sequence, conditioned on the values of the elements in all other positions of the provided sequence.
protected  String[] stringTagsAt(int pos)
           
 ArrayList<TaggedWord> tagSentence(List<? extends HasWord> s)
          Tags the sentence s by running maxent model.
protected  void writeTagsAndErrors(String[] finalTags, PrintFile pf, Dictionary wrongWords, boolean verboseResults)
          Write the tagging and note any errors (if pf != null) and accumulate global statistics.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

VERBOSE

protected static final boolean VERBOSE
See Also:
Constant Field Values

naTag

protected static final String naTag
See Also:
Constant Field Values

DBG

protected static final boolean DBG
See Also:
Constant Field Values

doDeterministicTagExpansion

protected static final boolean doDeterministicTagExpansion
See Also:
Constant Field Values

kBestSize

protected static final int kBestSize
See Also:
Constant Field Values

tagSeparator

protected final String tagSeparator

encoding

protected final String encoding

pairs

protected final PairsHolder pairs

sent

protected List<String> sent

origWords

protected List<HasWord> origWords

size

protected int size

correctTags

protected String[] correctTags

finalTags

protected String[] finalTags

localScores

protected volatile Map<String,double[]> localScores

localContextScores

protected volatile double[][] localContextScores

maxentTagger

protected final MaxentTagger maxentTagger
Constructor Detail

TestSentence

public TestSentence(MaxentTagger maxentTagger)

TestSentence

public TestSentence(MaxentTagger maxentTagger,
                    String[] s,
                    String[] correctTags,
                    PrintFile pf,
                    Dictionary wrongWords,
                    boolean verboseResults)
Method Detail

tagSentence

public ArrayList<TaggedWord> tagSentence(List<? extends HasWord> s)
Tags the sentence s by running maxent model. Returns a Sentence of TaggedWord objects.

Parameters:
s - Input sentence (List). This isn't changed.
Returns:
Tagged sentence

revert

protected void revert(int prevSize)

init

protected void init()

calculateProbs

protected void calculateProbs(double[][][] probabilities)
calculateProbs puts log probs of taggings in the probabilities array.

Parameters:
probabilities - Array with indices sent size, k best size, numTags

writeTagsAndErrors

protected void writeTagsAndErrors(String[] finalTags,
                                  PrintFile pf,
                                  Dictionary wrongWords,
                                  boolean verboseResults)
Write the tagging and note any errors (if pf != null) and accumulate global statistics.

Parameters:
finalTags - Chosen tags for sentence
pf - File to write tagged output to (can be null, then no output; at present it is non-null iff the debug property is set)
wrongWords - Dictionary to accumulate wrong word counts in (cannot be null)

initializeScorer

protected void initializeScorer()

cleanUpScorer

protected void cleanUpScorer()
clean-up after the scorer


append

protected String[] append(String[] tags,
                          String word)

getHistories

protected double[] getHistories(String[] tags,
                                History h)

length

public int length()
Specified by:
length in interface SequenceModel
Returns:
the length of the sequences modeled by this SequenceModel

leftWindow

public int leftWindow()
Description copied from interface: SequenceModel
How many positions to the left a position is dependent on.

Specified by:
leftWindow in interface SequenceModel
Returns:
the size of the left window used by this sequence model

rightWindow

public int rightWindow()
Description copied from interface: SequenceModel
How many positions to the right a position is dependent on.

Specified by:
rightWindow in interface SequenceModel
Returns:
the size of the right window used by this sequence model

getPossibleValues

public int[] getPossibleValues(int pos)
Description copied from interface: SequenceModel
0...leftWindow-1 etc are null, leftWindow...length+leftWindow-1 are words, length+leftWindow...length+leftWindow+rightWindow-1 are null;

Specified by:
getPossibleValues in interface SequenceModel
Parameters:
pos - the position
Returns:
the set of possible int values at this position, as an int array

scoreOf

public double scoreOf(int[] tags,
                      int pos)
Description copied from interface: SequenceModel
Computes the unnormalized log conditional distribution over values of the element at position pos in the sequence, conditioned on the values of the elements in all other positions of the provided sequence.

Specified by:
scoreOf in interface SequenceModel
Parameters:
tags - the sequence containing the rest of the values to condition on
pos - the position of the element to give a distribution for
Returns:
the log score of the token at the specified position in the sequence

scoreOf

public double scoreOf(int[] sequence)
Description copied from interface: SequenceModel
Computes the score assigned by this model to the whole sequence. Typically this will be an unnormalized probability in log space (since the probabilities are small).

Specified by:
scoreOf in interface SequenceModel
Parameters:
sequence - the sequence to compute a score for
Returns:
the score for the sequence

scoresOf

public double[] scoresOf(int[] tags,
                         int pos)
Description copied from interface: SequenceModel
Computes the unnormalized log conditional distribution over values of the element at position pos in the sequence, conditioned on the values of the elements in all other positions of the provided sequence.

Specified by:
scoresOf in interface SequenceModel
Parameters:
tags - the sequence containing the rest of the values to condition on
pos - the position of the element to give a distribution for
Returns:
the scores of the possible tokens at the specified position in the sequence

stringTagsAt

protected String[] stringTagsAt(int pos)


Stanford NLP Group