edu.stanford.nlp.international.arabic.pipeline

Class IBMMTArabicDataset

java.lang.Object

edu.stanford.nlp.international.arabic.pipeline.IBMMTArabicDataset

All Implemented Interfaces:

Dataset

public class IBMMTArabicDataset
extends java.lang.Object
implements Dataset

Applies the same orthographic transformations developed for ATB parse trees to flat MT input. This data set escapes IBM Arabic (for example, it removes explicit clitic markings).

NOTE: This class expects UTF-8 input (not Buckwalter)

Author:

Spence Green

Nested Class Summary

Nested classes/interfaces inherited from interface edu.stanford.nlp.trees.treebank.Dataset

Dataset.Encoding

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.Set<java.lang.String>	configuredOptions
protected IBMArabicEscaper	escaper
protected java.util.regex.Pattern	fileNameNormalizer
protected Mapper	lexMapper
protected java.lang.String	outFileName
protected java.util.List<java.io.File>	pathsToData
protected java.util.Set<java.lang.String>	requiredOptions
protected java.lang.StringBuilder	toStringBuilder

Constructor Summary

Constructors

Constructor and Description
IBMMTArabicDataset()

Method Summary

Modifier and Type	Method and Description
void	build() Generic method for loading, processing, and writing a dataset.
java.util.List<java.lang.String>	getFilenames() Returns the filenames written by Dataset.build().
boolean	setOptions(java.util.Properties opts) Sets options for a dataset.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

lexMapper

protected Mapper lexMapper

pathsToData

protected final java.util.List<java.io.File> pathsToData

outFileName

protected java.lang.String outFileName

fileNameNormalizer

protected final java.util.regex.Pattern fileNameNormalizer

escaper

protected final IBMArabicEscaper escaper

configuredOptions

protected final java.util.Set<java.lang.String> configuredOptions

requiredOptions

protected final java.util.Set<java.lang.String> requiredOptions

toStringBuilder

protected final java.lang.StringBuilder toStringBuilder

Constructor Detail

IBMMTArabicDataset

public IBMMTArabicDataset()

Method Detail

build

public void build()

Description copied from interface: Dataset

Generic method for loading, processing, and writing a dataset.

Specified by:

build in interface Dataset

getFilenames

public java.util.List<java.lang.String> getFilenames()

Description copied from interface: Dataset

Returns the filenames written by Dataset.build().

Specified by:

getFilenames in interface Dataset

Returns:

A collection of filenames

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

setOptions

public boolean setOptions(java.util.Properties opts)

Description copied from interface: Dataset

Sets options for a dataset.

Specified by:

setOptions in interface Dataset

Parameters:

opts - A map from parameter types defined in ConfigParser to values

Returns:

true if opts contains all required options. false, otherwise.