Informatics Report Series



Related Pages

Report (by Number) Index
Report (by Date) Index
Author Index
Institute Index

Title:Learning User Simulations for Information State Update Dialogue Systems
Authors: Kallirroi Georgila ; Oliver Lemon ; James Henderson
Date:Sep 2005
Publication Title:Proceedings of the 9th European Conference on Speech Communication and Technology (INTERSPEECH - EUROSPEECH)
Publication Type:Conference Paper Publication Status:Published
This paper describes and compares two methods for simulating user behaviour in spoken dialogue systems. User simulations are important for automatic dialogue strategy learning and the evaluation of competing strategies. Our methods are designed for use with Information State Update (ISU)-based dialogue systems. The first method is based on supervised learning using linear feature combination and a normalised exponential output function. The user is modelled as a stochastic process which selects user actions (<speech act, task> pairs) based on features of the current dialogue state, which encodes the whole history of the dialogue. The second method uses n-grams of <speech act, task> pairs, restricting the length of the history considered by the order of the n-gram. Both models were trained and evaluated on a subset of the COMMUNICATOR corpus, to which we added annotations for user actions and Information States. The model based on linear feature combination has a perplexity of 2.08 whereas the best n-gram (4-gram) has a perplexity of 3.58. Each one of the user models ran against a system policy trained on the same corpus with a method similar to the one used for our linear feature combination model. The quality of the simulated dialogues produced was then measured as a function of the filled slots, confirmed slots, and number of actions performed by the system in each dialogue. In this experiment both the linear feature combination model and the best n-grams (5-gram and 4-gram) produced similar quality simulated dialogues.
Links To Paper
1st link
Bibtex format
author = { Kallirroi Georgila and Oliver Lemon and James Henderson },
title = {Learning User Simulations for Information State Update Dialogue Systems},
book title = {Proceedings of the 9th European Conference on Speech Communication and Technology (INTERSPEECH - EUROSPEECH)},
year = 2005,
month = {Sep},
url = {},

Home : Publications : Report 

Please mail <> with any changes or corrections.
Unless explicitly stated otherwise, all material is copyright The University of Edinburgh