edu.stanford.nlp.process

Class WordTokenFactory

java.lang.Object

edu.stanford.nlp.process.WordTokenFactory

All Implemented Interfaces:

LexedTokenFactory<Word>

public class WordTokenFactory
extends java.lang.Object
implements LexedTokenFactory<Word>

Constructs a Word from a String. This is the default TokenFactory for PTBLexer. It discards the positional information.

Author:

Jenny Finkel

Constructor Summary

Constructors

Constructor and Description
WordTokenFactory()

Method Summary

Modifier and Type	Method and Description
Word	makeToken(java.lang.String str, int begin, int length) Constructs a token (of arbitrary type) from a String and its position in the underlying text.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WordTokenFactory

public WordTokenFactory()

Method Detail

makeToken

public Word makeToken(java.lang.String str,
                      int begin,
                      int length)

Description copied from interface: LexedTokenFactory

Constructs a token (of arbitrary type) from a String and its position in the underlying text. The int arguments are used just to record token character offsets in an underlying text. This method does not take a substring of str.

Specified by:

makeToken in interface LexedTokenFactory<Word>

Parameters:

str - The String extracted by the lexer.

begin - The offset in the document of the first character in this string.

length - The number of characters the string takes up in the document.

Returns:

The token of type T.