Srini Narayanan
ICSI Berkeley


Simulation Semantics: A Computational Framework Linking Language, Cognition and Action.



Abstract

The UCB/ICSI NTL (http://www.icsi.berkeley.edu/NTL) project is an ongoing attempt to model language behavior in a way that is both neurally plausible and computationally practical. A range of converging evidence from Cognitive Linguistics, Psychology and Neuroscience suggests that language understanding involves embodied enactment which we call "simulation semantics". Simulation semantics hypothesizes the mind as "simulating" the world while functioning in it. The "simulation" takes linguistic (and other sensory) input together with general knowledge and makes inferences to interpret the input and guide response. Monitoring the state of the world, drawing inferences, and acting jointly constitute a dynamic ongoing interactive process. This can be effectively modeled in a formalism (CPRM) combining structured Probablistic Models with extended Petri Nets. This talk reports on a computational realization of the simulation semantics hypothesis and preliminary results on applying the model to a Question Answering task.

Joint work with Jerome Feldman, George Lakoff, Nancy Chang, and the NTL group at Berkeley. The application to Question Answering is joint work with Sanda Harabagiu and her group at the University of Texas, Dallas and is supported under the ARDA AQUAINT program.