Title: Need-driven Affect Modeling for Spoken Conversational Agents: Design and Evaluation

Speaker: Juan Manuel Montero

Date: Friday, 20th of June 2014

One barrier to the creation of Spoken Conversational Agents (SCAs) has been the lack of methods for detecting and modelling emotions in a task-independent way. This seminar focuses on the design and evaluation of affective SCAs, from the rule-based speech understanding, automatic detection of the state of the user and the computation of internal emotional variables of the agent in each dialogue turn, to the expression of affect using emotional Text-To-Speech synthesis in Spanish.

The first module in the proposed architecture is an automatic affect predictor, offline-trained using data from the previous evaluation of a non-emotional SCA, a HiFi-control agent. The classifier is not based on the detection of contentment and frustration from voice, but from general performance indicators computed by the dialogue manager and the speech recognizer.

The core of the proposed architecture is an emotional model based on the Appraisal Theory and driven by quantifiable needs. The equations of the model have been tested in the HiFi SCA host, through a series of user studies with 70 subjects, conducted in a real-time environment. The results show that not only affective SCAs were preferred by the users, but also users´ frustration can be mitigated by the benefits of a user-adaptive exploitation of emotion, even when interacting with a lower-performance SCA.