edu.stanford.nlp.parser.shiftreduce

Class ShiftReduceParser

java.lang.Object

edu.stanford.nlp.parser.common.ParserGrammar

edu.stanford.nlp.parser.shiftreduce.ShiftReduceParser

All Implemented Interfaces:

ParserQueryFactory, java.io.Serializable, java.util.function.Function<java.util.List<? extends HasWord>,Tree>

public class ShiftReduceParser
extends ParserGrammar
implements java.io.Serializable

A shift-reduce constituency parser. Overview and description available at https://nlp.stanford.edu/software/srparser.shtml

Author:

John Bauer

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ShiftReduceParser(ShiftReduceOptions op)
ShiftReduceParser(ShiftReduceOptions op, PerceptronModel model)

Method Summary

Modifier and Type	Method and Description
static java.util.List<Tree>	binarizeTreebank(java.lang.Iterable<Tree> treebank, Options op)
static ShiftReduceOptions	buildTrainingOptions(java.lang.String tlppClass, java.lang.String[] args)
static boolean	checkLeafBranching(Tree tree) If an internal node goes directly to a leaf, that is an illegal tree.
static boolean	checkRootTransition(Tree tree) Some trees in the English datasets have a binary transition at the top, which we don't like as it teaches the parser to sometimes make binary transitions in normal trees
java.lang.String[]	defaultCoreNLPFlags() Returns a set of options which should be set by default when used in corenlp.
java.util.List<Tree>	filterTreebank(Treebank treebank) Filters any trees which are obviously unacceptable for the sr parser training.
static java.util.Set<java.lang.String>	findKnownStates(java.util.List<Tree> binarizedTrees)
static void	findKnownStates(Tree tree, java.util.Set<java.lang.String> knownStates)
java.util.List<Eval>	getExtraEvals() TODO: add an eval which measures transition accuracy?
Options	getOp()
java.util.List<ParserQueryEval>	getParserQueryEvals() Return a list of Eval-style objects which care about the whole ParserQuery, not just the finished tree
TreebankLangParserParams	getTLPParams()
static State	initialStateFromGoldTagTree(Tree tree)
static State	initialStateFromTaggedSentence(java.util.List<? extends HasWord> words)
java.util.Set<java.lang.String>	knownStates() Return an unmodifiableSet containing the known states (including binarization)
static ShiftReduceParser	loadModel(java.lang.String path, java.lang.String... extraFlags)
static void	main(java.lang.String[] args)
Tree	parse(java.util.List<? extends HasWord> sentence) Parses the list of HasWord.
Tree	parse(java.lang.String sentence) Will parse the text in sentence as if it represented a single sentence by first processing it with a tokenizer.
ParserQuery	parserQuery()
Tree	parseTree(java.util.List<? extends HasWord> sentence) Similar to parse(), but instead of returning an X tree on failure, returns null.
java.util.List<Tree>	readBinarizedTreebank(java.lang.String treebankPath, java.io.FileFilter treebankFilter)
Treebank	readTreebank(java.lang.String treebankPath, java.io.FileFilter treebankFilter)
static void	redoTags(java.util.List<Tree> trees, Tagger tagger, int nThreads)
static void	redoTags(Tree tree, Tagger tagger)
boolean	requiresTags() The model requires text to be pretagged
void	saveModel(java.lang.String path)
void	setOptionFlags(java.lang.String... flags)
java.util.Set<java.lang.String>	tagSet() Return the Set of POS tags used in the model.
TreebankLanguagePack	treebankLanguagePack()

Methods inherited from class edu.stanford.nlp.parser.common.ParserGrammar

apply, lemmatize, lemmatize, loadModelFromZip, loadTagger, tokenize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.function.Function

andThen, compose, identity

Constructor Detail

ShiftReduceParser

public ShiftReduceParser(ShiftReduceOptions op)

ShiftReduceParser

public ShiftReduceParser(ShiftReduceOptions op,
                         PerceptronModel model)

Method Detail

getOp

public Options getOp()

Specified by:

getOp in class ParserGrammar

getTLPParams

public TreebankLangParserParams getTLPParams()

Specified by:

getTLPParams in class ParserGrammar

treebankLanguagePack

public TreebankLanguagePack treebankLanguagePack()

Specified by:

treebankLanguagePack in class ParserGrammar

defaultCoreNLPFlags

public java.lang.String[] defaultCoreNLPFlags()

Description copied from class: ParserGrammar

Returns a set of options which should be set by default when used in corenlp. For example, the English PCFG/RNN models want -retainTmpSubcategories, and the ShiftReduceParser models may want -beamSize 4 depending on how they were trained.
TODO: right now completely hardcoded, should be settable as a training time option

Specified by:

defaultCoreNLPFlags in class ParserGrammar

knownStates

public java.util.Set<java.lang.String> knownStates()

Return an unmodifiableSet containing the known states (including binarization)

tagSet

public java.util.Set<java.lang.String> tagSet()

Return the Set of POS tags used in the model.

requiresTags

public boolean requiresTags()

Description copied from class: ParserGrammar

The model requires text to be pretagged

Specified by:

requiresTags in class ParserGrammar

parserQuery

public ParserQuery parserQuery()

Specified by:

parserQuery in interface ParserQueryFactory

Specified by:

parserQuery in class ParserGrammar

parse

public Tree parse(java.lang.String sentence)

Description copied from class: ParserGrammar

Will parse the text in sentence as if it represented a single sentence by first processing it with a tokenizer.

Overrides:

parse in class ParserGrammar

parse

public Tree parse(java.util.List<? extends HasWord> sentence)

Description copied from class: ParserGrammar

Parses the list of HasWord. If the parse fails for some reason, an X tree is returned instead of barfing.

Specified by:

parse in class ParserGrammar

Parameters:

sentence - The input sentence (a List of words)

Returns:

A Tree that is the parse tree for the sentence. If the parser fails, a new Tree is synthesized which attaches all words to the root.

parseTree

public Tree parseTree(java.util.List<? extends HasWord> sentence)

Description copied from class: ParserGrammar

Similar to parse(), but instead of returning an X tree on failure, returns null.

Specified by:

parseTree in class ParserGrammar

getExtraEvals

public java.util.List<Eval> getExtraEvals()

TODO: add an eval which measures transition accuracy?

Specified by:

getExtraEvals in class ParserGrammar

getParserQueryEvals

public java.util.List<ParserQueryEval> getParserQueryEvals()

Description copied from class: ParserGrammar

Return a list of Eval-style objects which care about the whole ParserQuery, not just the finished tree

Specified by:

getParserQueryEvals in class ParserGrammar

initialStateFromGoldTagTree

public static State initialStateFromGoldTagTree(Tree tree)

initialStateFromTaggedSentence

public static State initialStateFromTaggedSentence(java.util.List<? extends HasWord> words)

buildTrainingOptions

public static ShiftReduceOptions buildTrainingOptions(java.lang.String tlppClass,
                                                      java.lang.String[] args)

readTreebank

public Treebank readTreebank(java.lang.String treebankPath,
                             java.io.FileFilter treebankFilter)

readBinarizedTreebank

public java.util.List<Tree> readBinarizedTreebank(java.lang.String treebankPath,
                                                  java.io.FileFilter treebankFilter)

checkLeafBranching

public static boolean checkLeafBranching(Tree tree)

If an internal node goes directly to a leaf, that is an illegal tree. Otherwise, accept the tree.
Example:

(ROOT (sentence (S (morfema.pronominal (PRON Se)) -----(sn (spec (PRON los)) grup.nom)----- (grup.verb (PROPN trago')) (sn (spec (PROPN la)) (grup.nom (DET tierra)))) (NOUN ....) (S (sadv (grup.adv (PROPN ya))) (neg (ADV no)) (grup.verb (ADV viven)) (sp (prep (VERB en)) (sn (grup.nom (ADP Cuba)))) (PROPN ...)) (conj (PUNCT y)) (S (sp (prep (CCONJ a)) (sn (grup.nom (ADP nadie)))) (sn (grup.nom (PRON le))) (grup.verb (PRON importa)) (sn (spec (VERB la)) (grup.nom (S (relatiu (DET que)) (grup.verb (PRON esten) (gerundi (VERB pasando))))))) (VERB ....)))

checkRootTransition

public static boolean checkRootTransition(Tree tree)

Some trees in the English datasets have a binary transition at the top, which we don't like as it teaches the parser to sometimes make binary transitions in normal trees

filterTreebank

public java.util.List<Tree> filterTreebank(Treebank treebank)

Filters any trees which are obviously unacceptable for the sr parser training.
Disallowed:

• trees where internal nodes go directly to leaves instead of preterminals.
• trees which don't start with a unary transition

binarizeTreebank

public static java.util.List<Tree> binarizeTreebank(java.lang.Iterable<Tree> treebank,
                                                    Options op)

findKnownStates

public static java.util.Set<java.lang.String> findKnownStates(java.util.List<Tree> binarizedTrees)

findKnownStates

public static void findKnownStates(Tree tree,
                                   java.util.Set<java.lang.String> knownStates)

redoTags

public static void redoTags(Tree tree,
                            Tagger tagger)

redoTags

public static void redoTags(java.util.List<Tree> trees,
                            Tagger tagger,
                            int nThreads)

setOptionFlags

public void setOptionFlags(java.lang.String... flags)

Specified by:

setOptionFlags in class ParserGrammar

loadModel

public static ShiftReduceParser loadModel(java.lang.String path,
                                          java.lang.String... extraFlags)

saveModel

public void saveModel(java.lang.String path)

main

public static void main(java.lang.String[] args)