Package edu.stanford.nlp.util

A collection of useful general-purpose utility classes.

See:
          Description

Interface Summary
Agenda Agenda: an interface for representing an Agenda (a list of things to be done).
DisjointSet Disjoint Set: It's the disjoint set interface.
FileProcessor Interface for a Visitor pattern for Files.
Filter Filter is an interface for predicate objects which respond to the accept method.
Heap Heap: It's the heap interface.
Internable Internable: Interface for objects which are to be interned.
Scored Scored Author: Dan Klein Date: 12/4/00 Interface for representing a scored things
Weighted Weighted.
XScored  
 

Class Summary
ArrayHeap ArrayHeap: Heap implementation.
ArrayMap ArrayMap: A map that is backed by an Array
ArrayUtils Static utility methods for operating on arrays.
Beam Implements a finite beam, taking a comparator (default is ScoredComparator.ASCENDING_COMPARATOR, the MAX object according to the comparator is the one to be removed) and a beam size on construction (default is 100).
CollectionUtils Collection of useful static methods for working with Collections.
Counter Specialized Map for storing numeric counts for objects.
Dbg This class now combines two sets of debugging calls: Chris': Centralize the debugging print-outs, and make them more efficient when turned off at runtime.
DisabledPreferences A do-nothing Preferences implementation so that we can avoid the hassles of the JVM Preference implementations.
DisabledPreferencesFactory Returns do-nothing Preferences implementation.
EntryValueComparator Comparator for values of Map entries.
FastDisjointSet Fast Disjoint Set Disjoint forest with path compression and union-by-rank.
FilePathProcessor The FilePathProcessor traverses a directory structure and applies the processFile method to files meeting some criterion.
Filters Filters contains some simple implementations of the Filter interface.
Index Index is a list which maps between an Object vocabulary and contiguous integer indices.
Interner Interner: Class for interning things.
MutableInteger A class for Integer objects that you can change.
Numberer Gives a unique integer serial numbers to a family of objects, identified by a name space.
Pair Pair: A Class for holding a pair of objects.
ScoredComparator ScoredComparator allows one to compare scored things.
ScoredObject Scored Object: Wrapper class for holding a scored object
ScoredPair Scored Pair: Class for holding a scored pair of objects
ScoredPriorityAgenda ScoredPriorityAgenda
Sets Utilities for sets
StackAgenda StackAgenda
StringUtils Stringtils is a class for random String things.
Timing A class for measuring how long things take.
UnorderedPair Unordered Pair Class for holding an unordered pair of objects
 

Package edu.stanford.nlp.util Description

A collection of useful general-purpose utility classes. Below is a selection of some of the most useful utility classes, along with a brief description and sample use. Consult the class comments for more details on any of these classes.

Counter

Specialized Map for storing numeric counts for objects. Makes it easy to get/set/increment the count of an object and find the max/argmax. Also makes it easy to prune counts above/below a threshold and get the total count of all or a subset of the objects. Exposes a Comparator that can sort the keySet or entrySet by count.

Some useful methods: argmax, averageCount, comparator, incrementCount, keysAbove(threshold), max, normalize, totalCount

Example: generate a unigram language model for a Document with low counts (<3 counts) pruned:

Counter wordProbs = new Counter();
for(int i = 0; i < document.size(); i++) {
    wordProbs.incrementCount(document.get(i));
}
wordProbs.removeAll(wordProbs.keysBelow(3)); // prune low counts
wordProbs.normalize(); // convert to probability distribution

Example: find the Integer param that yields the best value of some computeScore method (that returns an int or double):

Counter paramScores = new Counter();
for(int param=0; param<10; param++) {
    paramScores.setCount(new Integer(param), computeScore(param));
}
Integer bestParam=(Integer)paramScores.argmax();

Filter

Interface to accept or reject Objects. A Filter implements boolean accept(Object obj). This can represent any binary predicate, such as "lowercase Strings", "numbers above a threshold", "trees where a VP dominates an NP and PP", and so on. Particularly useful in conjunction with Filters, which contains some basic filters as well as a method for filtering an array of Objects or a Collection. Another example is Counter's totalCount(Filter), which returns the sum of all counts in the Counter whose keys pass the filter.

Filters

Static class with some useful Filter implementations and utility methods for working with Filters. Contains Filters that always accept or reject, Filters that accept or reject an Object if it's in a given Collection, as well as several composite Filters. Contains methods for creating a new Filter that is the AND/OR of two Filters, or the NOT of a Filter. You can make a Filter that runs a given Appliable on all Objects before comparing them--this is useful when you have a collection of complex objects and you want to accept/reject based on one of their sub-objects or method values.  Finally, you can filter an Object[] through a Filter to return a new Object[] with only the accepeted values, or retainAll elements in a Collection that pass a Filter.

Some useful methods: andFilter(Filter, Filter), collectionAcceptFilter(Collection), filter(Object[], Filter)retainAll(Collection, Filter), transformedFilter(Filter, Appliable)

Example: Filter an array of Strings to retain only those with length less than 10:

Filter filter = new Filter() {
    public boolean accept(Object obj) {
        return (((String)obj).length < 10);
    }
}
String[] shortStrings = (String[])Filters.filter(allStrings, filter);

EntryValueComparator

Comparator for sorting Map keys and entries. If you use the empty Constructor, this Comparator will compare Map.Entry objects by comparing their values. If you pass a Map into the constructor, the Comparator can sort either the Map's keySet or entrySet. You can also pass an ascending flag to optionally reverse natural sorting order.

Sort a Map's keys by their values (descending order):

List keys = new ArrayList(map.keySet());
Collections.sort(keys, new EntryValueComparator(map, false));

Sort a Map's entries by their values (normal order):

List entries = new ArrayList(map.entrySet());
Collections.sort(entries, new EntryValueComparator());

Index

List that also maintains a constant-time reverse-lookup of indices for its Objects. Often one uses a List to associate a unique index with each Object (e.g. controlled vocbulary, feature map, etc.). Index offers constant-time performance for both index -> Object (get) and Object -> index (indexOf) as well as for contains(Object). Otherwise it behaves like a normal list. Index also supports lock() and unlock() to ensure that it's only modified when desired. Another useful method is int[] indices(List elems), which maps each elem to its index.

Some useful methods: add(Object), contains(Object), get(index), indexOf(Object), lock()

StringUtils

Static class with lots of useful String manipulation and formatting methods. Many of these methods will be familiar to perl users: join, split, trim, find, lookingAt, and matches. There are also useful methods for padding Strings/Objects with spaces on the right or left for printing even-width table columns: leftPad, pad. Finally, there are convenience methods for reading in all the text in a File or at a URL: slurpFile, slurpURL, as well as a method for making a "clean" filename from a String (where all spaces are turned into hyphens and non-alphanum chars become underscores): fileNameClean.

Example: print a comma-separated list of numbers:

System.out.println(StringUtils.pad(nums, ", "));

Example: print a 2D array of numbers with 8-char cells:

for(int i = 0; i < nums.length; i++) {
    for(int j = 0; j < nums[i].length; j++) {
        System.out.print(StringUtils.leftPad(nums[i][j], 8));
    }
    System.out.println();
}

Example: get a List of lines in a file (ignoring blank lines):

String fileContents = StringUtils.slurpFile(new File("filename"));
List lines = StringUtils.split(fileContents, "[\r\n]+");

Timing

Static class for measuring how long something takes to execute. To use, call startTime before running the code in question. Call tick to print an intermediate update, and endTime to finish the timing and print the result. You can optionally pass a descriptive string and PrintStream to tick and endTime for more control over what gets printed where.

Example: time reading in a big file and transforming it:

Timing.startTime();
String bigFileContents = StringUtils.slurpFile(bigFile);
Timing.tick("read in big file", System.err);
String output = costlyTransform(bigFileContents);
Timing.endTime("transformed big file", System.err);

Other packages with some useful utilies

edu.stanford.nlp.io
Contains some useful classes for traversing file systems to get lists of files, writing encoded output, and so on.
edu.stanford.nlp.process
Contains many useful text-filtering classes (they work on Documents from the dbm package).
edu.stanford.nlp.stats
Contains some useful classes for tracking statistics (counts) and performing various calculations (e.g. precision/recall)
edu.stanford.nlp.swing
Contains utilities for working with Swing GUIs, e.g. adding icons to your buttons, representing a GUI for properties, adding undo/redo support, adding smart text selection, etc.
edu.stanford.nlp.web
Contains some classes for doing programmatic web searches and parsing web pages.

Questionable classes in util

Internable - better version in oldparser?

Numberer - duplicate of Index?

XScored - ??



Stanford NLP Group