Work

Quantitative Evaluation of User Simulation Techniques for Spoken Dialogue Systems

Year: 2005
Type: conference-paper
Abstract: The lack of suitable training and testing data is currently a major roadblock in applying machine-learning techniques to dialogue management.Stochastic modelling of real users has been suggested as a solution to this problem, but to date few of the proposed models have been quantitatively evaluated on real data.Indeed, there are no established criteria for such an evaluation.This paper presents a systematic approach to testing user simulations and assesses the most prominent domain-independent techniques using a large DARPA Communicator corpus of human-computer dialogues.We show that while recent advances have led to significant improvements in simulation quality, simple statistical metrics are still sufficient to discern synthetic from real dialogues.
Language: English
Open Access status: hybrid
OpenAlex ID: W2603612888
FWCI: 9.474
Cites: 9
Cited by: 168
Related to: 9