Conversational Models powered by large pre-trained language models exhibit an impressive ability to deliver fluent and natural-sounding responses. Despite this performance, these models are fitful and can often generate unverifiable or factually incorrect statements, a phenomenon commonly called hallucination. In the first part of this talk, I will present a neural refinement approach that uses facts supplied by a knowledge graph to reduce entity-based hallucinations in generated responses. Second, I will discuss the origin of hallucinations in conversational models and will present a new benchmark for faithful information-seeking dialogue which drastically enhances faithfulness and other dialogue qualities. And lastly, I will briefly introduce the BEGIN benchmark designed to evaluate attribution in knowledge-grounded dialogue systems. Through a comprehensive evaluation study on BEGIN, I will show that a broad set of existing automatic metrics do not reliably distinguish attributable abstractive responses from unattributable ones, and perform substantially worse when the knowledge source is longer.
Nouha Dziri is a Ph.D candidate at the University of Alberta working within the Alberta Machine Intelligence Institute under the supervision of Osmar Zaiane. Her research interests lie in building trustworthy conversational models from three perspectives: modelling, data, and evaluation. She has interned at Google Research, Microsoft Research, and Mila. Her work has been published in top-tier venues including TACL, NAACL and EMNLP. She actively serves as a reviewer for NLP conferences, journals, and workshops and was recognized among the best reviewers at ACL 2021. She is also a proponent of diversity and gives several talks to inspire females to pursue careers in STEM.