As models like BERT, T5, and GPT-* have grown larger, more powerful, and more widespread, we've also grown from seeing them as black boxes to having some understanding of what they learn and how they behave. Viewing these models as contextual encoders, I'll present a few of our recent findings about what kind of knowledge they capture, how this knowledge is organized, and what happens to it when they're fine-tuned on a downstream task. Finally, I'll talk about some of our efforts to put this understanding into practice via the Language Interpretability Tool (LIT).
Ian Tenney is a software engineer on the Language team at Google Research. His research focuses on interpretability and analysis of deep NLP models, particularly on how they encode linguistic structure and how this structure informs model behavior. He holds an M.S. in Computer Science and a B.S. in Physics from Stanford.