Foundations of Statistical Natural Language Processing

Christopher D. Manning and Hinrich Schütze

Chapter 2: Mathematical Foundations

Teaching materials

• Powerpoint slides by Barbara Rosario (UC Berkeley) [reproduced with permission of the author].
• Acrobat slides on KL-divergence and cross entropy

Links of interest

• StatLib (CMU) is a longstanding archive of free statistical software, but not always the easiest place to find things.
• StatCodes nominally targets astronomers, but is a well-laid out site with a fairly extensive coverage of statistical and visualization software.
• HyperStat Online -- an online statistics course.
• There's lots of information about Claude Shannon on the web. There are some brief biographies of varying quality and originality (bio-1, bio-2, bio-3, bio-4), a copy of his 1948 paper A mathematical theory of communication, and information about Shannon's juggling theorem.