|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectedu.stanford.nlp.parser.lexparser.BaseUnknownWordModel
edu.stanford.nlp.parser.lexparser.EnglishUnknownWordModel
public class EnglishUnknownWordModel
This is a basic unknown word model for English. It supports 5 different
types of feature modeling; see getSignature(String, int)
.
Implementation note: the contents of this class tend to overlap somewhat
with ArabicUnknownWordModel
and were originally included in BaseLexicon
.
Field Summary | |
---|---|
protected boolean |
smartMutation
|
protected int |
unknownPrefixSize
|
protected int |
unknownSuffixSize
|
protected java.lang.String |
wordClassesFile
|
Fields inherited from class edu.stanford.nlp.parser.lexparser.BaseUnknownWordModel |
---|
NULL_ITW, nullTag, nullWord, tagHash, tagIndex, trainOptions, unknown, unknownLevel, unSeenCounter, useFirst, useGT, VERBOSE, wordIndex |
Constructor Summary | |
---|---|
EnglishUnknownWordModel(Options op,
Lexicon lex,
Index<java.lang.String> wordIndex,
Index<java.lang.String> tagIndex)
This constructor creates an UWM with empty data structures. |
|
EnglishUnknownWordModel(Options op,
Lexicon lex,
Index<java.lang.String> wordIndex,
Index<java.lang.String> tagIndex,
ClassicCounter<IntTaggedWord> unSeenCounter)
|
Method Summary | |
---|---|
java.lang.String |
getSignature(java.lang.String word,
int loc)
This routine returns a String that is the "signature" of the class of a word. |
int |
getSignatureIndex(int index,
int sentencePosition,
java.lang.String word)
Returns the index of the signature of the word numbered wordIndex, where the signature is the String representation of unknown word features. |
float |
score(IntTaggedWord iTW,
int loc,
double c_Tseen,
double total,
double smooth,
java.lang.String word)
Currently we don't consider loc or the other parameters in determining score in the default implementation; only English uses them. |
double |
scoreProbTagGivenWordSignature(IntTaggedWord iTW,
int loc,
double smooth,
java.lang.String word)
Calculate P(Tag|Signature) with Bayesian smoothing via just P(Tag|Unknown) |
Methods inherited from class edu.stanford.nlp.parser.lexparser.BaseUnknownWordModel |
---|
addTagging, getLexicon, getUnknownLevel, score, scoreGT, setUnknownLevel, unSeenCounter |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected final boolean smartMutation
protected final int unknownSuffixSize
protected final int unknownPrefixSize
protected final java.lang.String wordClassesFile
Constructor Detail |
---|
public EnglishUnknownWordModel(Options op, Lexicon lex, Index<java.lang.String> wordIndex, Index<java.lang.String> tagIndex, ClassicCounter<IntTaggedWord> unSeenCounter)
public EnglishUnknownWordModel(Options op, Lexicon lex, Index<java.lang.String> wordIndex, Index<java.lang.String> tagIndex)
Method Detail |
---|
public float score(IntTaggedWord iTW, int loc, double c_Tseen, double total, double smooth, java.lang.String word)
BaseUnknownWordModel
score
in interface UnknownWordModel
score
in class BaseUnknownWordModel
iTW
- An IntTaggedWord pairing a word and POS tagloc
- The position in the sentence. In the default implementation
this is used only for unknown words to change their
probability distribution when sentence initial. Now,
a negative value c_Tseen
- Total count of this tag (on seen words) in trainingtotal
- Total count of word tokens in trainingsmooth
- Weighting on prior P(T|U) in estimateword
- The word itself; useful so we don't look it up in the index
public double scoreProbTagGivenWordSignature(IntTaggedWord iTW, int loc, double smooth, java.lang.String word)
scoreProbTagGivenWordSignature
in interface UnknownWordModel
scoreProbTagGivenWordSignature
in class BaseUnknownWordModel
public int getSignatureIndex(int index, int sentencePosition, java.lang.String word)
getSignatureIndex
in interface UnknownWordModel
getSignatureIndex
in class BaseUnknownWordModel
public java.lang.String getSignature(java.lang.String word, int loc)
getSignature
in interface UnknownWordModel
getSignature
in class BaseUnknownWordModel
word
- The word to make a signature forloc
- Its position in the sentence (mainly so sentence-initial
capitalized words can be treated differently)
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |