Package edu.stanford.nlp.util

A collection of useful general-purpose utility classes.

See:
          Description

Interface Summary
Agenda Agenda: an interface for representing an Agenda (a list of things to be done).
DisjointSet Disjoint Set: It's the disjoint set interface.
FileProcessor Interface for a Visitor pattern for Files.
Filter Filter is an interface for predicate objects which respond to the accept method.
Heap Heap: It's the heap interface.
PriorityQueue Created by IntelliJ IDEA.
Scored Scored Author: Dan Klein Date: 12/4/00 Interface for representing a scored things
Weighted Weighted.
XScored  
 

Class Summary
AbstractIterator Iterator with remove defined to throw UnsupportedOperationException .
ArrayHeap ArrayHeap: Heap implementation.
ArrayMap ArrayMap: A map that is backed by an Array
ArrayUtils Static utility methods for operating on arrays.
Beam Implements a finite beam, taking a comparator (default is ScoredComparator.ASCENDING_COMPARATOR, the MAX object according to the comparator is the one to be removed) and a beam size on construction (default is 100).
BinaryHeapPriorityQueue PriorityQueue with explicit double priority values.
Characters  
CircleList A CircleList wraps another list, presenting an apparently infinite list by wrapping around all requests for an item, as if the list were circular.
CollectionFactory Factory for vending Collections.
CollectionFactory.ArrayListFactory  
CollectionFactory.HashSetFactory  
CollectionUtils Collection of useful static methods for working with Collections.
CollectionValuedMap User: Teg Grenager Date: Jan 12, 2004 Time: 9:15:53 AM
CompositionMap A map which contains a pointer to another map, and refers all operations to the second map.
ConcatenationIterator Iterator that represents the concatenation of two other iterators.
Counter A specialized kind of hash table (or map) for storing numeric counts for objects.
Counters Static methods for operating on Counters.
DeltaCollectionValuedMap User: Teg Grenager Date: Jan 14, 2004 Time: 10:40:57 AM
DeltaCounter A Counter that is maintained as a delta off another Counter.
DeltaMap A Map which wraps an original Map, and only stores the changes (deltas) from the original Map.
DisabledPreferences A do-nothing Preferences implementation so that we can avoid the hassles of the JVM Preference implementations.
DisabledPreferencesFactory Returns do-nothing Preferences implementation.
Distribution Immutable class for representing normalized, smoothed discrete distributions from Counters.
EditDistance  
EntryValueComparator Comparator designed for the values of Map entries.
FastDisjointSet Fast Disjoint Set Disjoint forest with path compression and union-by-rank.
FilePathProcessor The FilePathProcessor traverses a directory structure and applies the processFile method to files meeting some criterion.
FilteredIterator Iterator that suppresses items in another iterator based on a filter function.
Filters Filters contains some simple implementations of the Filter interface.
GeneralizedCounter A class for keeping double counts of Lists of a prespecified length.
IdentityWrapper Wrapper to make an object use identity equality and hashcode.
Index An Index is a list which maps between an Object vocabulary and contiguous integer indices.
Interner For interning (canonicalizing) things.
IntersectionIterator Iterator over the union of two sets.
Iterators Contains convenience methods, and utilities for java.util.Iterators
LongestCommonSubstring Similarity metric based on longest-common-substrings between two strings.
MapFactory Factory for vending Maps.
MapFactory.HashMapFactory  
MapFactory.IdentityHashMapFactory  
MapFactory.TreeMapFactory  
MapFactory.WeakHashMapFactory  
Maps Utilities for Maps, including inverting, composing, and support for list/set values.
MutableDouble A class for Double objects that you can change.
MutableInteger A class for Integer objects that you can change.
NullComparator Comparator that says every pair of objects is tied.
Numberer Gives unique integer serial numbers to a family of objects, identified by a name space.
PaddedList A PaddedList wraps another list, presenting an apparently infinite list by padding outside the real confines of the list with a default value.
Pair Pair: A Class for holding a pair of objects.
PrettyDouble  
ScoredComparator ScoredComparator allows one to compare scored things.
ScoredObject Scored Object: Wrapper class for holding a scored object
ScoredPair Scored Pair: Class for holding a scored pair of objects
ScoredPriorityAgenda ScoredPriorityAgenda
Sets Utilities for sets
StackAgenda StackAgenda
StringUtils Stringtils is a class for random String things.
Timing A class for measuring how long things take.
Triple User: Teg Grenager Date: Feb 20, 2004 Time: 9:44:12 AM
UnionIterator Iterator over the union of two sets.
UnorderedPair Unordered Pair Class for holding an unordered pair of objects
 

Package edu.stanford.nlp.util Description

A collection of useful general-purpose utility classes. Below is a selection of some of the most useful utility classes, along with a brief description and sample use. Consult the class comments for more details on any of these classes.

Counter

Specialized Map for storing numeric counts for objects. Makes it easy to get/set/increment the count of an object and find the max/argmax. Also makes it easy to prune counts above/below a threshold and get the total count of all or a subset of the objects. Exposes a Comparator that can sort the keySet or entrySet by count.

Some useful methods: argmax, averageCount, comparator, incrementCount, keysAbove(threshold), max, normalize, totalCount

Example: generate a unigram language model for a Document with low counts (<3 counts) pruned:

Counter wordProbs = new Counter();
for(int i = 0; i < document.size(); i++) {
    wordProbs.incrementCount(document.get(i));
}
wordProbs.removeAll(wordProbs.keysBelow(3)); // prune low counts
wordProbs.normalize(); // convert to probability distribution

Example: find the Integer param that yields the best value of some computeScore method (that returns an int or double):

Counter paramScores = new Counter();
for(int param=0; param<10; param++) {
    paramScores.setCount(new Integer(param), computeScore(param));
}
Integer bestParam=(Integer)paramScores.argmax();

Filter

Interface to accept or reject Objects. A Filter implements boolean accept(Object obj). This can represent any binary predicate, such as "lowercase Strings", "numbers above a threshold", "trees where a VP dominates an NP and PP", and so on. Particularly useful in conjunction with Filters, which contains some basic filters as well as a method for filtering an array of Objects or a Collection. Another example is Counter's totalCount(Filter), which returns the sum of all counts in the Counter whose keys pass the filter.

Filters

Static class with some useful Filter implementations and utility methods for working with Filters. Contains Filters that always accept or reject, Filters that accept or reject an Object if it's in a given Collection, as well as several composite Filters. Contains methods for creating a new Filter that is the AND/OR of two Filters, or the NOT of a Filter. You can make a Filter that runs a given Appliable on all Objects before comparing them--this is useful when you have a collection of complex objects and you want to accept/reject based on one of their sub-objects or method values.  Finally, you can filter an Object[] through a Filter to return a new Object[] with only the accepeted values, or retainAll elements in a Collection that pass a Filter.

Some useful methods: andFilter(Filter, Filter), collectionAcceptFilter(Collection), filter(Object[], Filter)retainAll(Collection, Filter), transformedFilter(Filter, Appliable)

Example: Filter an array of Strings to retain only those with length less than 10:

Filter filter = new Filter() {
    public boolean accept(Object obj) {
        return (((String)obj).length < 10);
    }
}
String[] shortStrings = (String[])Filters.filter(allStrings, filter);

EntryValueComparator

Comparator for sorting Map keys and entries. If you use the empty Constructor, this Comparator will compare Map.Entry objects by comparing their values. If you pass a Map into the constructor, the Comparator can sort either the Map's keySet or entrySet. You can also pass an ascending flag to optionally reverse natural sorting order.

Sort a Map's keys by their values (descending order):

List keys = new ArrayList(map.keySet());
Collections.sort(keys, new EntryValueComparator(map, false));

Sort a Map's entries by their values (normal order):

List entries = new ArrayList(map.entrySet());
Collections.sort(entries, new EntryValueComparator());

Index

List that also maintains a constant-time reverse-lookup of indices for its Objects. Often one uses a List to associate a unique index with each Object (e.g. controlled vocbulary, feature map, etc.). Index offers constant-time performance for both index -> Object (get) and Object -> index (indexOf) as well as for contains(Object). Otherwise it behaves like a normal list. Index also supports lock() and unlock() to ensure that it's only modified when desired. Another useful method is int[] indices(List elems), which maps each elem to its index.

Some useful methods: add(Object), contains(Object), get(index), indexOf(Object), lock()

StringUtils

Static class with lots of useful String manipulation and formatting methods. Many of these methods will be familiar to perl users: join, split, trim, find, lookingAt, and matches. There are also useful methods for padding Strings/Objects with spaces on the right or left for printing even-width table columns: leftPad, pad. Finally, there are convenience methods for reading in all the text in a File or at a URL: slurpFile, slurpURL, as well as a method for making a "clean" filename from a String (where all spaces are turned into hyphens and non-alphanum chars become underscores): fileNameClean.

Example: print a comma-separated list of numbers:

System.out.println(StringUtils.pad(nums, ", "));

Example: print a 2D array of numbers with 8-char cells:

for(int i = 0; i < nums.length; i++) {
    for(int j = 0; j < nums[i].length; j++) {
        System.out.print(StringUtils.leftPad(nums[i][j], 8));
    }
    System.out.println();
}

Example: get a List of lines in a file (ignoring blank lines):

String fileContents = StringUtils.slurpFile(new File("filename"));
List lines = StringUtils.split(fileContents, "[\r\n]+");

Timing

Static class for measuring how long something takes to execute. To use, call startTime before running the code in question. Call tick to print an intermediate update, and endTime to finish the timing and print the result. You can optionally pass a descriptive string and PrintStream to tick and endTime for more control over what gets printed where.

Example: time reading in a big file and transforming it:

Timing.startTime();
String bigFileContents = StringUtils.slurpFile(bigFile);
Timing.tick("read in big file", System.err);
String output = costlyTransform(bigFileContents);
Timing.endTime("transformed big file", System.err);

Other packages with some useful utilies

edu.stanford.nlp.io
Contains some useful classes for traversing file systems to get lists of files, writing encoded output, and so on.
edu.stanford.nlp.process
Contains many useful text-filtering classes (they work on Documents from the dbm package).
edu.stanford.nlp.stats
Contains some useful classes for tracking statistics (counts) and performing various calculations (e.g. precision/recall)
edu.stanford.nlp.swing
Contains utilities for working with Swing GUIs, e.g. adding icons to your buttons, representing a GUI for properties, adding undo/redo support, adding smart text selection, etc.
edu.stanford.nlp.web
Contains some classes for doing programmatic web searches and parsing web pages.

Questionable classes in util

Numberer - duplicate of Index?

XScored - ??



Stanford NLP Group