Package edu.stanford.nlp.util

A collection of useful general-purpose utility classes.

See:
          Description

Interface Summary
Filter<T> Filter is an interface for predicate objects which respond to the accept method.
PriorityQueue<E> A Set that also represents an ordering of its elements, and responds quickly to add(), changePriority(), removeFirst(), and getFirst() method calls.
 

Class Summary
AbstractIterator<E> Iterator with remove() defined to throw an UnsupportedOperationException.
ArrayMap<K,V> ArrayMap: A map that is backed by an Array
BinaryHeapPriorityQueue<E> PriorityQueue with explicit double priority values.
EntryValueComparator Comparator designed for the values of Map entries.
Index<E> An Index is a collection that maps between an Object vocabulary and a contiguous non-negative integer index beginning (inclusively) at 0.
IntPair  
IntQuadruple  
IntTriple  
IntTuple A tuple of int.
IntUni Just a single integer
MapFactory A factory class for vending different sorts of Maps.
MutableDouble A class for Double objects that you can change.
MutableInteger A class for Integer objects that you can change.
Pair<T1,T2> Pair: A Class for holding a pair of objects.
Sets Utilities for sets
StringUtils StringUtils is a class for random String things, including output formatting and command line argument parsing.
Timing A class for measuring how long things take.
XMLUtils Class XMLUtils
XMLUtils.XMLTag  
 

Package edu.stanford.nlp.util Description

A collection of useful general-purpose utility classes. Below is a selection of some of the most useful utility classes, along with a brief description and sample use. Consult the class comments for more details on any of these classes.

edu.stanford.nlp.util.Counter

Specialized Map for storing numeric counts for objects. Makes it easy to get/set/increment the count of an object and find the max/argmax. Also makes it easy to prune counts above/below a threshold and get the total count of all or a subset of the objects. Exposes a Comparator that can sort the keySet or entrySet by count.

Some useful methods: argmax, averageCount, comparator, incrementCount, keysAbove(threshold), max, normalize, totalCount

Example: generate a unigram language model for a Document with low counts (<3 counts) pruned:

Counter wordProbs = new Counter();
for(int i = 0; i < document.size(); i++) {
    wordProbs.incrementCount(document.get(i));

wordProbs.removeAll(wordProbs.keysBelow(3)); // prune low counts
wordProbs.normalize(); // convert to probability distribution

Example: find the Integer param that yields the best value of some computeScore method (that returns an int or double):

Counter paramScores = new Counter();
for(int param=0; param<10; param++) {
    paramScores.setCount(new Integer(param), computeScore(param));

Integer bestParam=(Integer)paramScores.argmax();

edu.stanford.nlp.util.Filter

Interface to accept or reject Objects. A Filter implements boolean accept(Object obj). This can represent any binary predicate, such as "lowercase Strings", "numbers above a threshold", "trees where a VP dominates an NP and PP", and so on. Particularly useful in conjunction with Filters, which contains some basic filters as well as a method for filtering an array of Objects or a Collection. Another example is Counter's totalCount(Filter), which returns the sum of all counts in the Counter whose keys pass the filter.

edu.stanford.nlp.util.Filters

Static class with some useful Filter implementations and utility methods for working with Filters. Contains Filters that always accept or reject, Filters that accept or reject an Object if it's in a given Collection, as well as several composite Filters. Contains methods for creating a new Filter that is the AND/OR of two Filters, or the NOT of a Filter. You can make a Filter that runs a given Appliable on all Objects before comparing them--this is useful when you have a collection of complex objects and you want to accept/reject based on one of their sub-objects or method values.  Finally, you can filter an Object[] through a Filter to return a new Object[] with only the accepeted values, or retainAll elements in a Collection that pass a Filter.

Some useful methods: andFilter(Filter, Filter), collectionAcceptFilter(Collection), filter(Object[], Filter)retainAll(Collection, Filter), transformedFilter(Filter, Appliable)

Example: Filter an array of Strings to retain only those with length less than 10:

Filter filter = new Filter() {
    public boolean accept(Object obj) {
        return (((String)obj).length < 10);
   

String[] shortStrings = (String[])Filters.filter(allStrings, filter);

edu.stanford.nlp.util.EntryValueComparator

Comparator for sorting Map keys and entries. If you use the empty Constructor, this Comparator will compare Map.Entry objects by comparing their values. If you pass a Map into the constructor, the Comparator can sort either the Map's keySet or entrySet. You can also pass an ascending flag to optionally reverse natural sorting order.

Sort a Map's keys by their values (descending order):

List keys = new ArrayList(map.keySet());
Collections.sort(keys, new EntryValueComparator(map, false));

Sort a Map's entries by their values (normal order):

List entries = new ArrayList(map.entrySet());
Collections.sort(entries, new EntryValueComparator());

edu.stanford.nlp.util.Index

List that also maintains a constant-time reverse-lookup of indices for its Objects. Often one uses a List to associate a unique index with each Object (e.g. controlled vocbulary, feature map, etc.). Index offers constant-time performance for both index -> Object (get) and Object -> index (indexOf) as well as for contains(Object). Otherwise it behaves like a normal list. Index also supports lock() and unlock() to ensure that it's only modified when desired. Another useful method is int[] indices(List elems), which maps each elem to its index.

Some useful methods: add(Object), contains(Object), get(index), indexOf(Object), lock()

edu.stanford.nlp.util.StringUtils

Static class with lots of useful String manipulation and formatting methods. Many of these methods will be familiar to perl users: join, split, trim, find, lookingAt, and matches. There are also useful methods for padding Strings/Objects with spaces on the right or left for printing even-width table columns: leftPad, pad. Finally, there are convenience methods for reading in all the text in a File or at a URL: slurpFile, slurpURL, as well as a method for making a "clean" filename from a String (where all spaces are turned into hyphens and non-alphanum chars become underscores): fileNameClean.

Example: print a comma-separated list of numbers:

System.out.println(StringUtils.pad(nums, ", "));

Example: print a 2D array of numbers with 8-char cells:

for(int i = 0; i < nums.length; i++) {
    for(int j = 0; j < nums[i].length; j++) {
        System.out.print(StringUtils.leftPad(nums[i][j], 8));
   
    System.out.println();

Example: get a List of lines in a file (ignoring blank lines):

String fileContents = StringUtils.slurpFile(new File("filename"));
List lines = StringUtils.split(fileContents, "[\r\n]+");

edu.stanford.nlp.util.Timing

Static class for measuring how long something takes to execute. To use, call startTime before running the code in question. Call tick to print an intermediate update, and endTime to finish the timing and print the result. You can optionally pass a descriptive string and PrintStream to tick and endTime for more control over what gets printed where.

Example: time reading in a big file and transforming it:

Timing.startTime();
String bigFileContents = StringUtils.slurpFile(bigFile);
Timing.tick("read in big file", System.err);
String output = costlyTransform(bigFileContents);
Timing.endTime("transformed big file", System.err);

Other packages with some useful utilies

edu.stanford.nlp.io
Contains some useful classes for traversing file systems to get lists of files, writing encoded output, and so on.
edu.stanford.nlp.process
Contains many useful text-filtering classes (they work on Documents from the dbm package).
edu.stanford.nlp.stats
Contains some useful classes for tracking statistics (counts) and performing various calculations (e.g. precision/recall)
edu.stanford.nlp.swing
Contains utilities for working with Swing GUIs, e.g. adding icons to your buttons, representing a GUI for properties, adding undo/redo support, adding smart text selection, etc.
edu.stanford.nlp.web
Contains some classes for doing programmatic web searches and parsing web pages.

Questionable classes in util

Numberer: this is sort of a duplicate of Index, but adds a level of namespaces on top. But it's widely used and doesn't quite seem worth removing.