edu.stanford.nlp.wordseg

Class MaxMatchSegmenter

java.lang.Object

edu.stanford.nlp.wordseg.MaxMatchSegmenter

All Implemented Interfaces:

WordSegmenter, java.io.Serializable

public class MaxMatchSegmenter
extends java.lang.Object
implements WordSegmenter

Lexicon-based segmenter. Uses dynamic programming to find a word segmentation that satisfies the following two preferences: (1) minimize the number of out-of-vocabulary (OOV) words; (2) if there are multiple segmentations with the same number of OOV words, then select the one that minimizes the number of segments. Note that MaxMatchSegmenter contains a greedy version of this algorithm. Note that the output segmentation may need to postprocessing for the segmentation of non-Chinese characters (e.g., punctuation, foreign names).

Author:

Michel Galley

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MaxMatchSegmenter.MatchHeuristic

Constructor Summary

Constructors

Constructor and Description
MaxMatchSegmenter()

Method Summary

Modifier and Type	Method and Description
void	finishTraining()
java.util.ArrayList<Word>	greedilySegmentWords(java.lang.String s) Returns a lexicon-based segmentation.
void	initializeTraining(double numTrees)
void	loadSegmenter(java.lang.String filename)
static void	main(java.lang.String[] args)
java.util.ArrayList<Word>	maxMatchSegmentation() Returns the lexicon-based segmentation that minimizes the number of words.
java.util.List<HasWord>	segment(java.lang.String s)
java.util.ArrayList<Word>	segmentWords(MaxMatchSegmenter.MatchHeuristic h) Returns the lexicon-based segmentation following heuristic h.
void	train(java.util.Collection<Tree> trees)
void	train(java.util.List<TaggedWord> sentence)
void	train(Tree tree)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MaxMatchSegmenter

public MaxMatchSegmenter()

Method Detail

initializeTraining

public void initializeTraining(double numTrees)

Specified by:

initializeTraining in interface WordSegmenter

train

public void train(java.util.Collection<Tree> trees)

Specified by:

train in interface WordSegmenter

train

public void train(Tree tree)

Specified by:

train in interface WordSegmenter

train

public void train(java.util.List<TaggedWord> sentence)

Specified by:

train in interface WordSegmenter

finishTraining

public void finishTraining()

Specified by:

finishTraining in interface WordSegmenter

loadSegmenter

public void loadSegmenter(java.lang.String filename)

Specified by:

loadSegmenter in interface WordSegmenter

segment

public java.util.List<HasWord> segment(java.lang.String s)

Specified by:

segment in interface WordSegmenter

maxMatchSegmentation

public java.util.ArrayList<Word> maxMatchSegmentation()

Returns the lexicon-based segmentation that minimizes the number of words.

Returns:

Segmented sentence.

segmentWords

public java.util.ArrayList<Word> segmentWords(MaxMatchSegmenter.MatchHeuristic h)
                                       throws java.lang.UnsupportedOperationException

Returns the lexicon-based segmentation following heuristic h. Note that buildSegmentationLattice must be run first. Two heuristics are currently available -- MINWORDS and MAXWORDS -- to respectively minimize and maximize the number of segment (where each segment is a lexicon word, if possible).

Parameters:

h - Heuristic to use for segmentation.

Returns:

Segmented sentence.

Throws:

java.lang.UnsupportedOperationException

See Also:

buildSegmentationLattice(java.lang.String)

greedilySegmentWords

public java.util.ArrayList<Word> greedilySegmentWords(java.lang.String s)

Returns a lexicon-based segmentation. At each position x in the input string, it attempts to find largest value y, so that [x,y] is part of the lexicon. Then, it tried to match more input from position y+1. This greedy algorithm (taken from edu.stanford.nlp.lexparser.MaxMatchSegmenter) has no theoretical guarantee, and it would be wise to use segmentWords instead.

Parameters:

s - Input (unsegmented) string.

Returns:

Segmented sentence.

main

public static void main(java.lang.String[] args)