edu.stanford.nlp.process

Class CoreLabelProcessor

java.lang.Object

edu.stanford.nlp.process.CoreLabelProcessor

Direct Known Subclasses:

CodepointCoreLabelProcessor, GermanTokenizerPostProcessor

public abstract class CoreLabelProcessor
extends java.lang.Object

Abstract class for processing a List<CoreLabel>. Example use case is merging on hyphens for German NER (process) and then restoring after classification (restore).

Constructor Summary

Constructors

Constructor and Description
CoreLabelProcessor()

Method Summary

Modifier and Type	Method and Description
abstract java.util.List<CoreLabel>	process(java.util.List<CoreLabel> tokens) Alter the tokenization of list of tokens (e.g.
abstract java.util.List<CoreLabel>	restore(java.util.List<CoreLabel> originalTokens, java.util.List<CoreLabel> processedTokens) Undo the tokenization changes of process, maintaining any tagging

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CoreLabelProcessor

public CoreLabelProcessor()

Method Detail

process

public abstract java.util.List<CoreLabel> process(java.util.List<CoreLabel> tokens)

Alter the tokenization of list of tokens (e.g. map split on hyphen to don't split)

restore

public abstract java.util.List<CoreLabel> restore(java.util.List<CoreLabel> originalTokens,
                                                  java.util.List<CoreLabel> processedTokens)

Undo the tokenization changes of process, maintaining any tagging