This talk is part of the NLP Seminar Series.

Humans Learn From Task Descriptions and So Should Our Models

Hinrich Schütze, LMU Munich
Date: 10:00am - 11:00am PT, Apr 15 2021
Venue: Zoom (link hidden)


In many types of human learning, task descriptions are a central ingredient. They are usually accompanied by a few examples, but there is very little human learning that is based on examples only. In contrast, the typical learning setup for NLP tasks lacks task descriptions and is supervised with 100s or 1000s of examples. This is even true for so-called few-shot learning, a term often applied to scenarios with tens of thousands of "shots".

Inspired by the GPT models, which also exploit task descriptions, we introduce Pattern-Exploiting Training (PET). PET reformulates task descriptions as cloze questions that can be effectively processed by pretrained language models. In contrast to GPT, PET combines task descriptions with supervised learning. We show that PET learns well from as little as ten training examples and outperforms GPT-3 on GLUE even though it has 99.9% fewer parameters.


Hinrich Schütze (PhD 1995, Stanford University) is Professor for Computational Linguistics and director of the Center for Information and Language Processing at the University of Munich (LMU Munich). Before moving to Munich in 2013, he taught at the University of Stuttgart. He worked on natural language processing and information retrieval technology at Xerox PARC, at several Silicon Valley startups and at Google 1995-2004 and 2008/9. He is a coauthor of Foundations of Statistical Natural Language Processing (with Chris Manning) and Introduction to Information Retrieval (with Chris Manning and Prabhakar Raghavan).