The rapid adoption of a new generation of voice-based conversational assistants such as Alexa, Google Assistant, and Siri presents new research challenges for conversational information seeking, including conversational search. This talk focuses on lessons learned developing reusable test collections for evaluating conversational search systems including the TREC Conversational Assistance Track (CAsT), and ClariQ in ConvAI3. We discuss our work on neural methods for conversational query understanding and rewriting. We describe a study of state-of-the-art models and present areas for improvement.
Dr. Jeff Dalton is an Assistant Professor in the School of Computing Science at the University of Glasgow where he leads the Glasgow Representation and Information Learning Lab (GRILL) (https://grilllab.ai). His research focuses on text understanding and conversational information seeking. He completed his Ph.D. at the University of Massachusetts Amherst in the Center for Intelligent Information Retrieval. Later in Google Research, he worked on Information Extraction as part of the Knowledge Discovery Team (Knowledge Vault) and on language understanding in the Assistant Response Ranking team. He is the lead organizer for the TREC Conversational Assistance Track (CAsT) (http://treccast.ai) and previously helped organize the Complex Answer Retrieval track. He is the recipient of a prestigious UKRI Turing AI Acceleration Fellowship on Neural Conversational Assistants and received research awards from Google, Amazon, and Bloomberg. He is the faculty advisor for the 2021/2022 Alexa Prize Taskbot challenge team, GRILLBot. He holds multiple patents in retrieval, information extraction, and question answering.
Carlos is an NLP PhD student at the University of Glasgow supervised by Jeff Dalton. His research interests lie at the intersection of question answering, semantic parsing and conversational systems where he extends neural networks to use symbolic tools as a means to make them more interpretable and sample efficient. During his PhD, Carlos helps organize and participate in TREC CAsT, and currently leads the GRILL Alexa team as one of nine global teams in the Amazon Alexa TaskBot Challenge.