edu.stanford.nlp.util

Class CollectionUtils

java.lang.Object

edu.stanford.nlp.util.CollectionUtils

public class CollectionUtils
extends Object

Collection of useful static methods for working with Collections. Includes methods to increment counts in maps and cast list/map elements to common types.

Author:

Joseph Smarr (jsmarr@stanford.edu)

Method Summary

Modifier and Type	Method and Description
static <T> void	addAll(Collection<T> collection, Iterable<? extends T> items) Add all the items from an iterable to a collection.
static double[]	asDoubleArray(Collection<Double> coll)
static int[]	asIntArray(Collection<Integer> coll)
static List<Double>	asList(double[] a)
static List<Integer>	asList(int[] a)
static <T> Set<T>	asSet(T[] o) Returns a new Set containing all the objects in the specified array.
static <T extends Comparable<T>> int	compareLists(List<T> list1, List<T> list2) Provides a consistent ordering over lists.
static <T> boolean	containsAny(Collection<T> collection, Collection<T> toCheck) if any item in toCheck is present in collection
static <T> boolean	containsObject(Collection<T> c, T o) Checks whether a Collection contains a specified Object.
static <T> Collection<T>	diff(Collection<T> list1, Collection<T> list2) Returns all objects in list1 that are not in list2
static <T> Set<T>	diffAsSet(Collection<T> list1, Collection<T> list2) Returns all objects in list1 that are not in list2
static <T> List<T>	filterAsList(Collection<? extends T> original, java.util.function.Predicate<? super T> f) Filters the objects in the collection according to the given Filter and returns a list
static <T> List<T>	flatten(Collection<List<T>> nestedList) combines all the lists in a collection to a single list
static <T,V> List<V>	getAll(Map<T,V> map, Collection<T> indices) Get all values corresponding to the indices (if they exist in the map).
static <T> int	getIndex(List<T> l, T o) Returns the index of the first occurrence in the list of the specified object, using object identity (==) not equality as the criterion for object presence.
static <T> int	getIndex(List<T> l, T o, int fromIndex) Returns the index of the first occurrence after the startIndex (exclusive) in the list of the specified object, using object equals function.
static <C extends Comparable<C>> Comparator<List<C>>	getListComparator()
static <K,V> Map<K,V>	getMapFromString(String s, Class<K> keyClass, Class<V> valueClass, MapFactory<K,V> mapFactory)
static <T> List<List<T>>	getNGrams(List<T> items, int minSize, int maxSize) Get all sub-lists of the given list of the given sizes.
static <T> List<List<T>>	getPrefixesAndSuffixes(List<T> items, int minSize, int maxSize, T paddingSymbol, boolean includePrefixes, boolean includeSuffixes) Get all prefix/suffix combinations from a list.
static <T> Set<T>	intersection(Set<T> set1, Set<T> set2)
static <T> boolean	isSubList(List<T> l1, List<? super T> l) Returns true iff l1 is a sublist of l (i.e., every member of l1 is in l, and for every e1 < e2 in l1, there is an e1 < e2 occurrence in l).
static <T> void	loadCollection(File file, Class<T> itemClass, Collection<T> collection) Adds the items from the file to the collection.
static <T> Collection<T>	loadCollection(File file, Class<T> c, CollectionFactory<T> cf)
static <T> void	loadCollection(String fileName, Class<T> itemClass, Collection<T> collection) Adds the items from the file to the collection.
static <T> Collection<T>	loadCollection(String filename, Class<T> c, CollectionFactory<T> cf)
static <T> List<T>	makeList(T... items) Returns a new List containing the given objects.
static <T extends Comparable<? super T>> int	maxIndex(List<T> list)
static <T,M> List<T>	mergeList(List<? extends T> list, Collection<M> matched, java.util.function.Function<M,Interval<Integer>> toIntervalFunc, java.util.function.Function<List<? extends T>,T> aggregator)
static <T> List<T>	mergeList(List<? extends T> list, List<? extends HasInterval<Integer>> matched, java.util.function.Function<List<? extends T>,T> aggregator)
static <T> List<T>	mergeListWithSortedMatched(List<? extends T> list, List<? extends HasInterval<Integer>> matched, java.util.function.Function<List<? extends T>,T> aggregator)
static <T> List<T>	mergeListWithSortedMatchedPreAggregated(List<? extends T> list, List<? extends T> matched, java.util.function.Function<T,Interval<Integer>> toIntervalFunc)
static <T> T	mode(Collection<T> values) Returns the mode in the Collection.
static <T> Set<T>	modes(Collection<T> values) Returns a list of all modes in the Collection.
static <T> List<Collection<T>>	partitionIntoFolds(List<T> values, int numFolds) Split a list into numFolds (roughly) equally sized folds.
static <T> boolean	removeObject(List<T> l, T o) Removes the first occurrence in the list of the specified object, using object identity (==) not equality as the criterion for object presence.
static <E> E	sample(List<E> l, Random r)
static <E> Collection<E>	sampleWithoutReplacement(Collection<E> c, int n) Samples without replacement from a collection.
static <E> Collection<E>	sampleWithoutReplacement(Collection<E> c, int n, Random r) Samples without replacement from a collection, using your own Random number generator.
static <E> Collection<E>	sampleWithReplacement(Collection<E> c, int n) Samples with replacement from a collection
static <E> Collection<E>	sampleWithReplacement(Collection<E> c, int n, Random r) Samples with replacement from a collection, using your own Random number generator
static <T extends Comparable<T>> List<T>	sorted(Iterable<T> items) Return the items of an Iterable as a sorted list.
static <T> List<T>	sorted(Iterable<T> items, Comparator<T> comparator) Return the items of an Iterable as a sorted list.
static <T> List<T>	toList(Iterable<T> items) Create a list out of the items in the Iterable.
static <T> Set<T>	toSet(Iterable<T> items) Create a set out of the items in the Iterable.
static <K,V> String	toVerticalString(Map<K,V> m)
static <T> Collection<Pair<Collection<T>,Collection<T>>>	trainTestFoldsForCV(List<T> values, int numFolds) Split a list into train, test pairs for use in k-fold crossvalidation.
static <T1,T2> List<T2>	transformAsList(Collection<? extends T1> original, java.util.function.Function<T1,? extends T2> f) Transforms the keyset of collection according to the given Function and returns a list
static <T1,T2> Set<T2>	transformAsSet(Collection<? extends T1> original, java.util.function.Function<T1,? extends T2> f) Transforms the keyset of collection according to the given Function and returns a set of the keys
static <T> Collection<T>	union(Collection<T> set1, Collection<T> set2)
static <T> Set<T>	unionAsSet(Collection<T>... sets)
static <T> Set<T>	unionAsSet(Collection<T> set1, Collection<T> set2)
static <ObjType,Hashable> Collection<ObjType>	uniqueNonhashableObjects(Collection<ObjType> objects, java.util.function.Function<ObjType,Hashable> customHasher) Makes it possible to uniquify a collection of objects which are normally non-hashable.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

asList

public static List<Integer> asList(int[] a)

asList

public static List<Double> asList(double[] a)

asIntArray

public static int[] asIntArray(Collection<Integer> coll)

asDoubleArray

public static double[] asDoubleArray(Collection<Double> coll)

makeList

public static <T> List<T> makeList(T... items)

Returns a new List containing the given objects.

asSet

public static <T> Set<T> asSet(T[] o)

Returns a new Set containing all the objects in the specified array.

intersection

public static <T> Set<T> intersection(Set<T> set1,
                                      Set<T> set2)

union

public static <T> Collection<T> union(Collection<T> set1,
                                      Collection<T> set2)

unionAsSet

public static <T> Set<T> unionAsSet(Collection<T> set1,
                                    Collection<T> set2)

unionAsSet

public static <T> Set<T> unionAsSet(Collection<T>... sets)

diff

public static <T> Collection<T> diff(Collection<T> list1,
                                     Collection<T> list2)

Returns all objects in list1 that are not in list2

Type Parameters:

T - Type of items in the collection

Parameters:

list1 - First collection

list2 - Second collection

Returns:

The collection difference list1 - list2

diffAsSet

public static <T> Set<T> diffAsSet(Collection<T> list1,
                                   Collection<T> list2)

Returns all objects in list1 that are not in list2

Type Parameters:

T - Type of items in the collection

Parameters:

list1 - First collection

list2 - Second collection

Returns:

The collection difference list1 - list2

loadCollection

public static <T> Collection<T> loadCollection(String filename,
                                               Class<T> c,
                                               CollectionFactory<T> cf)
                                        throws Exception

Parameters:

filename - the path to the file to load the List from

c - the Class to instantiate each member of the List. Must have a String constructor.

Throws:

Exception

loadCollection

public static <T> Collection<T> loadCollection(File file,
                                               Class<T> c,
                                               CollectionFactory<T> cf)
                                        throws Exception

Parameters:

file - the file to load the List from

c - the Class to instantiate each member of the List. Must have a String constructor.

Throws:

Exception

loadCollection

public static <T> void loadCollection(String fileName,
                                      Class<T> itemClass,
                                      Collection<T> collection)
                               throws NoSuchMethodException,
                                      InstantiationException,
                                      IllegalAccessException,
                                      InvocationTargetException,
                                      IOException

Adds the items from the file to the collection.

Type Parameters:

T - The type of the items.

Parameters:

fileName - The name of the file from which items should be loaded.

itemClass - The class of the items (must have a constructor that accepts a String).

collection - The collection to which items should be added.

Throws:

NoSuchMethodException

InstantiationException

IllegalAccessException

InvocationTargetException

IOException

loadCollection

public static <T> void loadCollection(File file,
                                      Class<T> itemClass,
                                      Collection<T> collection)
                               throws NoSuchMethodException,
                                      InstantiationException,
                                      IllegalAccessException,
                                      InvocationTargetException,
                                      IOException

Adds the items from the file to the collection.

Type Parameters:

T - The type of the items.

Parameters:

file - The file from which items should be loaded.

itemClass - The class of the items (must have a constructor that accepts a String).

collection - The collection to which items should be added.

Throws:

NoSuchMethodException

InstantiationException

IllegalAccessException

InvocationTargetException

IOException

getMapFromString

public static <K,V> Map<K,V> getMapFromString(String s,
                                              Class<K> keyClass,
                                              Class<V> valueClass,
                                              MapFactory<K,V> mapFactory)
                                       throws ClassNotFoundException,
                                              NoSuchMethodException,
                                              IllegalAccessException,
                                              InvocationTargetException,
                                              InstantiationException

Throws:

ClassNotFoundException

NoSuchMethodException

IllegalAccessException

InvocationTargetException

InstantiationException

containsObject

public static <T> boolean containsObject(Collection<T> c,
                                         T o)

Checks whether a Collection contains a specified Object. Object equality (==), rather than .equals(), is used.

removeObject

public static <T> boolean removeObject(List<T> l,
                                       T o)

Removes the first occurrence in the list of the specified object, using object identity (==) not equality as the criterion for object presence. If this list does not contain the element, it is unchanged.

Parameters:

l - The List from which to remove the object

o - The object to be removed.

Returns:

Whether or not the List was changed.

getIndex

public static <T> int getIndex(List<T> l,
                               T o)

Returns the index of the first occurrence in the list of the specified object, using object identity (==) not equality as the criterion for object presence. If this list does not contain the element, return -1.

Parameters:

l - The List to find the object in.

o - The sought-after object.

Returns:

Whether or not the List was changed.

getIndex

public static <T> int getIndex(List<T> l,
                               T o,
                               int fromIndex)

Returns the index of the first occurrence after the startIndex (exclusive) in the list of the specified object, using object equals function. If this list does not contain the element, return -1.

Parameters:

l - The List to find the object in.

o - The sought-after object.

fromIndex - The start index

Returns:

Whether or not the List was changed.

sampleWithoutReplacement

public static <E> Collection<E> sampleWithoutReplacement(Collection<E> c,
                                                         int n)

Samples without replacement from a collection.

Parameters:

c - The collection to be sampled from

n - The number of samples to take

Returns:

a new collection with the sample

sampleWithoutReplacement

public static <E> Collection<E> sampleWithoutReplacement(Collection<E> c,
                                                         int n,
                                                         Random r)

Samples without replacement from a collection, using your own Random number generator.

Parameters:

c - The collection to be sampled from

n - The number of samples to take

r - the random number generator

Returns:

a new collection with the sample

sample

public static <E> E sample(List<E> l,
                           Random r)

sampleWithReplacement

public static <E> Collection<E> sampleWithReplacement(Collection<E> c,
                                                      int n)

Samples with replacement from a collection

Parameters:

c - The collection to be sampled from

n - The number of samples to take

Returns:

a new collection with the sample

sampleWithReplacement

public static <E> Collection<E> sampleWithReplacement(Collection<E> c,
                                                      int n,
                                                      Random r)

Samples with replacement from a collection, using your own Random number generator

Parameters:

c - The collection to be sampled from

n - The number of samples to take

r - the random number generator

Returns:

a new collection with the sample

isSubList

public static <T> boolean isSubList(List<T> l1,
                                    List<? super T> l)

Returns true iff l1 is a sublist of l (i.e., every member of l1 is in l, and for every e1 < e2 in l1, there is an e1 < e2 occurrence in l).

toVerticalString

public static <K,V> String toVerticalString(Map<K,V> m)

compareLists

public static <T extends Comparable<T>> int compareLists(List<T> list1,
                                                         List<T> list2)

Provides a consistent ordering over lists. First compares by the first element. If that element is equal, the next element is considered, and so on.

getListComparator

public static <C extends Comparable<C>> Comparator<List<C>> getListComparator()

sorted

public static <T extends Comparable<T>> List<T> sorted(Iterable<T> items)

Return the items of an Iterable as a sorted list.

Type Parameters:

T - The type of items in the Iterable.

Parameters:

items - The collection to be sorted.

Returns:

A list containing the same items as the Iterable, but sorted.

sorted

public static <T> List<T> sorted(Iterable<T> items,
                                 Comparator<T> comparator)

Return the items of an Iterable as a sorted list.

Type Parameters:

T - The type of items in the Iterable.

Parameters:

items - The collection to be sorted.

Returns:

A list containing the same items as the Iterable, but sorted.

toList

public static <T> List<T> toList(Iterable<T> items)

Create a list out of the items in the Iterable.

Type Parameters:

T - The type of items in the Iterable.

Parameters:

items - The items to be made into a list.

Returns:

A list consisting of the items of the Iterable, in the same order.

toSet

public static <T> Set<T> toSet(Iterable<T> items)

Create a set out of the items in the Iterable.

Type Parameters:

T - The type of items in the Iterable.

Parameters:

items - The items to be made into a set.

Returns:

A set consisting of the items from the Iterable.

addAll

public static <T> void addAll(Collection<T> collection,
                              Iterable<? extends T> items)

Add all the items from an iterable to a collection.

Type Parameters:

T - The type of items in the iterable and the collection

Parameters:

collection - The collection to which the items should be added.

items - The items to add to the collection.

getNGrams

public static <T> List<List<T>> getNGrams(List<T> items,
                                          int minSize,
                                          int maxSize)

Get all sub-lists of the given list of the given sizes. For example:

 List<String> items = Arrays.asList("a", "b", "c", "d");
 System.out.println(CollectionUtils.getNGrams(items, 1, 2));
 

would print out:

 [[a], [a, b], [b], [b, c], [c], [c, d], [d]]
 

Type Parameters:

T - The type of items contained in the list.

Parameters:

items - The list of items.

minSize - The minimum size of an ngram.

maxSize - The maximum size of an ngram.

Returns:

All sub-lists of the given sizes.

getPrefixesAndSuffixes

public static <T> List<List<T>> getPrefixesAndSuffixes(List<T> items,
                                                       int minSize,
                                                       int maxSize,
                                                       T paddingSymbol,
                                                       boolean includePrefixes,
                                                       boolean includeSuffixes)

Get all prefix/suffix combinations from a list. It can extract just prefixes, just suffixes, or prefixes and suffixes of the same length. For example:

 List<String> items = Arrays.asList("a", "b", "c", "d");
 System.out.println(CollectionUtils.getPrefixesAndSuffixes(items, 1, 2, null, true, true));
 

would print out:

 [[d], [a], [a, d], [d, c], [a, b], [a, b, c, d]]
 

and

 List<String> items2 = Arrays.asList("a");
 System.out.println(CollectionUtils.getPrefixesAndSuffixes(items2, 1, 2, null, true, true));
 

would print:

 [[a], [a], [a, a], [a, null], [a, null], [a, null, a, null]]
 

Type Parameters:

T - The type of items contained in the list.

Parameters:

items - The list of items.

minSize - The minimum length of a prefix/suffix span (should be at least 1)

maxSize - The maximum length of a prefix/suffix span

paddingSymbol - Symbol to be included if we run out of bounds (e.g. if items has size 3 and we try to extract a span of length 4).

includePrefixes - whether to extract prefixes

includeSuffixes - whether to extract suffixes

Returns:

All prefix/suffix combinations of the given sizes.

mergeList

public static <T,M> List<T> mergeList(List<? extends T> list,
                                      Collection<M> matched,
                                      java.util.function.Function<M,Interval<Integer>> toIntervalFunc,
                                      java.util.function.Function<List<? extends T>,T> aggregator)

mergeList

public static <T> List<T> mergeList(List<? extends T> list,
                                    List<? extends HasInterval<Integer>> matched,
                                    java.util.function.Function<List<? extends T>,T> aggregator)

mergeListWithSortedMatched

public static <T> List<T> mergeListWithSortedMatched(List<? extends T> list,
                                                     List<? extends HasInterval<Integer>> matched,
                                                     java.util.function.Function<List<? extends T>,T> aggregator)

mergeListWithSortedMatchedPreAggregated

public static <T> List<T> mergeListWithSortedMatchedPreAggregated(List<? extends T> list,
                                                                  List<? extends T> matched,
                                                                  java.util.function.Function<T,Interval<Integer>> toIntervalFunc)

flatten

public static <T> List<T> flatten(Collection<List<T>> nestedList)

combines all the lists in a collection to a single list

uniqueNonhashableObjects

public static <ObjType,Hashable> Collection<ObjType> uniqueNonhashableObjects(Collection<ObjType> objects,
                                                                              java.util.function.Function<ObjType,Hashable> customHasher)

Makes it possible to uniquify a collection of objects which are normally non-hashable. Alternatively, it lets you define an alternate hash function for them for limited-use hashing.

containsAny

public static <T> boolean containsAny(Collection<T> collection,
                                      Collection<T> toCheck)

if any item in toCheck is present in collection

Parameters:

collection -

toCheck -

partitionIntoFolds

public static <T> List<Collection<T>> partitionIntoFolds(List<T> values,
                                                         int numFolds)

Split a list into numFolds (roughly) equally sized folds. The earlier folds may have one more item in them than later folds.

trainTestFoldsForCV

public static <T> Collection<Pair<Collection<T>,Collection<T>>> trainTestFoldsForCV(List<T> values,
                                                                                    int numFolds)

Split a list into train, test pairs for use in k-fold crossvalidation. This returns a list of numFold (train, test) pairs where each train list will contain (numFolds-1)/numFolds of the original values and the test list will contain the remaining 1/numFolds of the original values.

modes

public static <T> Set<T> modes(Collection<T> values)

Returns a list of all modes in the Collection. (If the Collection has multiple items with the highest frequency, all of them will be returned.)

mode

public static <T> T mode(Collection<T> values)

Returns the mode in the Collection. If the Collection has multiple modes, this method picks one arbitrarily.

transformAsSet

public static <T1,T2> Set<T2> transformAsSet(Collection<? extends T1> original,
                                             java.util.function.Function<T1,? extends T2> f)

Transforms the keyset of collection according to the given Function and returns a set of the keys

transformAsList

public static <T1,T2> List<T2> transformAsList(Collection<? extends T1> original,
                                               java.util.function.Function<T1,? extends T2> f)

Transforms the keyset of collection according to the given Function and returns a list

filterAsList

public static <T> List<T> filterAsList(Collection<? extends T> original,
                                       java.util.function.Predicate<? super T> f)

Filters the objects in the collection according to the given Filter and returns a list

getAll

public static <T,V> List<V> getAll(Map<T,V> map,
                                   Collection<T> indices)

Get all values corresponding to the indices (if they exist in the map).

Parameters:

map - Any map from T to V

indices - A collection of indices of type T

Returns:

The corresponding list of values of type V

maxIndex

public static <T extends Comparable<? super T>> int maxIndex(List<T> list)