Stanford Named Entity Recognizer (NER)

About | Questions | Mailing lists | Download | Extensions | Models | Online demo | Release history | FAQ

About

Stanford NER is a Java implementation of a Named Entity Recognizer. Named Entity Recognition (NER) labels sequences of words in a text which are the names of things, such as person and company names, or gene and protein names. It comes with well-engineered feature extractors for Named Entity Recognition, and many options for defining feature extractors. Included with the download are good named entity recognizers for English, particularly for the 3 classes (PERSON, ORGANIZATION, LOCATION), and we also make available on this page various other models for different languages and circumstances, including models trained on just the CoNLL 2003 English training data. The distributional similarity features in some models improve performance but the models require considerably more memory.

Stanford NER is also known as CRFClassifier. The software provides a general implementation of (arbitrary order) linear chain Conditional Random Field (CRF) sequence models. That is, by training your own models, you can actually use this code to build sequence models for any task. (CRF models were pioneered by Lafferty, McCallum, and Pereira (2001); see Sutton and McCallum (2006) or Sutton and McCallum (2010) for more comprehensible introductions.)

The CRF code is by Jenny Finkel. The feature extractors are by Dan Klein, Christopher Manning, and Jenny Finkel. Much of the documentation and usability is due to Anna Rafferty. The CRF sequence models provided here do not precisely correspond to any published paper, but the correct paper to cite for the software is:

Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling. Proceedings of the 43nd Annual Meeting of the Association for Computational Linguistics (ACL 2005), pp. 363-370. http://nlp.stanford.edu/~manning/papers/gibbscrf3.pdf

The software provided here is similar to the baseline local+Viterbi model in that paper, but adds new distributional similarity based features (in the -distSim classifiers). The big models were trained on a mixture of CoNLL, MUC-6, MUC-7 and ACE named entity corpora, and as a result the models are fairly robust across domains.

You can look at a Powerpoint Introduction to NER and the Stanford NER package [ppt] [pdf] or the FAQ, which has some information on training models. Further documentation is provided in the included README and in the javadocs.

Stanford NER is available for download, licensed under the GNU General Public License (v2 or later). Source is included. The package includes components for command-line invocation (look at the shell scripts and batch files included in the download), running as a server (look at NERServer in the sources jar file), and a Java API (look at the simple examples in the NERDemo.java file included in the download, and then at the javadocs). Stanford NER code is dual licensed (in a similar manner to MySQL, etc.). Open source licensing is under the full GPL, which allows many free uses. For distributors of proprietary software, commercial licensing is available. If you don't need a commercial license, but would like to support maintenance of these tools, we welcome gifts.


Questions

There is also a list of Frequently Asked Questions (with answers!). Additional questions, feedback, and bug reports/fixes can be sent to our mailing lists.


Mailing Lists

We have 3 mailing lists for the Stanford Named Entity Recognizer, all of which are shared with other JavaNLP tools (with the exclusion of the parser). Before writing, please check to see if your question has been answered in the FAQ. Each address is at @lists.stanford.edu:

  1. java-nlp-user This is the best list to post to in order to ask questions, make announcements, or for discussion among JavaNLP users. You have to subscribe to be able to use it. Join the list via this webpage or by emailing java-nlp-user-join@lists.stanford.edu. (Leave the subject and message body empty.) You can also look at the list archives.
  2. java-nlp-announce This list will be used only to announce new versions of Stanford JavaNLP tools. So it will be very low volume (expect 1-3 messages a year). Join the list via this webpage or by emailing java-nlp-announce-join@lists.stanford.edu. (Leave the subject and message body empty.)
  3. java-nlp-support This list goes only to the software maintainers. It's a good address for licensing questions, etc. For general use and support questions, you're better off joining and using java-nlp-user. You cannot join java-nlp-support, but you can mail questions to java-nlp-support@lists.stanford.edu.

Download

Download Stanford Named Entity Recognizer version 3.4.1

The download is a 90M zipped file (mainly consisting of classifier data objects). If you unpack that file, you should have everything needed for English NER (or use as a general CRF). It includes batch files for running under Windows or Unix/Linux/MacOSX, a simple GUI, and the ability to run as a server. Stanford NER requires Java v1.6+. If you want to use Stanford NER for other languages, you'll also need to download model files for those languages; see further below.


Extensions: Packages by others using Stanford NER

For some (computer) languages, there are more up-to-date interfaces to Stanford NER available by using it inside Stanford CoreNLP, and you are better off getting those from the CoreNLP page an using thema....


Models

Included with Stanford NER are a 4 class model trained for CoNLL, a 7 class model trained for MUC, and a 3 class model trained on both data sets for the intersection of those class sets.

3 class:Location, Person, Organization
4 class:Location, Person, Organization, Misc
7 class: Time, Location, Organization, Person, Money, Percent, Date

These models each use distributional similarity features, which provide some performance gain at the cost of increasing their size and runtime. Also available are the same models missing those features.

Models with no distsim features


Also available, as part of a package of caseless models for several of our tools, are caseless versions of these same three models. You can either unpack the jar file or add it to the classpath; if you add the jar file to the classpath, you can then load the models from the path edu/stanford/nlp/models/.... You can run jar -t to get the list of files in the jar file.

Models which ignore capitalization


Furthermore, a pair of German models are available. For citation and other information relating to the German classifiers, please see Sebastian Pado's German NER page .

German NER



As of version 3.4.1, we have a Spanish model available for NER. It is included in the Spanish corenlp models jar.

Spanish CoreNLP models



We also provide Chinese models built from the Ontonotes Chinese named entity data. There are two models, one using distributional similarity clusters and one without. These are designed to be run on word-segmented Chinese. So, if you want to use these on normal Chinese text, you will first need to run Stanford Word Segmenter or some other Chinese word segmenter, and then run NER on the output of that!
Chinese NER


Online Demo

We have an online demo of several of our NER models. Special thanks to Dat Hoang, who provided the initial version. Note that the online demo demonstrates single CRF models; in order to see the effect of the time annotator or the combined models, see CoreNLP.


Release History


VersionDateDescription
3.4.1 2014-08-27 Added Spanish models
3.4 2014-06-16 Fix serialization of new models
3.3.1 2014-01-04 Bugfix release
3.3.0 2013-11-12 Updated for compatibility
3.2.0 2013-06-20 Improved line by line handling
1.2.8 2013-04-04 -nthreads option
1.2.7 2012-11-11 Add Chinese model, include Wikipedia data in 3-class English model
1.2.6 2012-07-09 Minor bug fixes
1.2.5 2012-05-22 Fix encoding issue
1.2.4 2012-04-07 Caseless versions of models supported
1.2.3 2012-01-06 Minor bug fixes
1.2.2 2011-09-14 Improved thread safety
1.2.1 2011-06-19 Models reduced in size but on average improved in accuracy (improved distsim clusters)
1.2 2011-05-16 Normal download includes 3, 4, and 7 class models. Updated for compatibility with other software releases.
1.1.1 2009-01-16 Minor bug and usability fixes, and changed API (in particular the methods to classify and output tagged text)
1.1 2008-05-07 Additional feature flags, various code updates
1.0 2006-09-18 Initial release