Large language models (LLMs) have undoubtedly transformed the landscape of NLP and artificial intelligence today. Despite their remarkable impact, these LLMs still have inherent limitations: they are costly to update, and tend to generate factually incorrect information, and easy to leak private data. In this talk, I will discuss recent progress and challenges towards building retrieval-based language models (augmenting LMs with an external, non-parametric retrieval component), which have shown promise to address the above fundamental limitations. The first half of the talk will discuss learning methods and neural architectures of retrieval-based language models, and the key challenges we are facing to scale up these models. In the second half, I will discuss how to train such models in privacy-sensitive domains, and whether having an external datastore would alleviate the privacy leakage problem.
Danqi Chen is an Assistant Professor of Computer Science at Princeton University and co-leads the Princeton NLP Group. Her recent research focuses on training, adapting and understanding large language models, and developing scalable and efficient NLP systems. Before joining Princeton, Danqi worked as a visiting scientist at Facebook AI Research. She received her Ph.D. from Stanford University (2018) and B.E. from Tsinghua University (2012), both in Computer Science. Her research was recognized by a Sloan Fellowship, an NSF CAREER award, a Samsung AI Researcher of the Year award, outstanding paper awards from ACL and EMNLP, and multiple industrial faculty awards.