edu.stanford.nlp.process
Interface LexedTokenFactory
- All Known Implementing Classes:
- WordTokenFactory
public interface LexedTokenFactory
Constructs a token (of arbitrary type) from a String and its position
in the underlying text.
Method Summary |
java.lang.Object |
makeToken(java.lang.String str,
int begin,
int length)
Constructs a token (of arbitrary type) from a String and its position
in the underlying text. |
makeToken
java.lang.Object makeToken(java.lang.String str,
int begin,
int length)
- Constructs a token (of arbitrary type) from a String and its position
in the underlying text.
- Parameters:
str
- The String extracted by the lexer.begin
- The offset in the document of the first character
in this string.length
- The number of characters the string takes up in
the document.