- Abstract:
-
Inference in Markov Decision Processes has recently received interest as a means to infer goals of an observed action, policy recognition, and also as a tool to compute policies. A particularly interesting aspect of the approach is that any existing inference technique in DBNs now becomes available for answering behavioral questions including those on continuous, factorial, or hierarchical state representations. Here we present an Expectation Maximization algorithm for computing optimal policies. Unlike previous approaches we can show that this actually optimizes the discounted expected future return for arbitrary reward functions and without assuming an ad hoc finite total time. The algorithm is generic in that any inference technique can be utilized in the E-step. We demonstrate this for exact inference on a discrete maze and Gaussian belief state propagation in continuous stochastic optimal control problems.
- Links To Paper
- 1st Link
- Bibtex format
- @InProceedings{EDI-INF-RR-0848,
- author = {
Mark Toussaint
and Amos Storkey
},
- title = {Probabilistic Inference for Solving Discrete and Continuous State Markov Decision Processes},
- book title = {Proceedings of 23nd International Conference on Machine Learning (ICML 2006)},
- year = 2006,
- url = {http://www.anc.ed.ac.uk/~amos/publications/ToussaintStorkey2006ProbabilisticInferenceSolvingMDPs.pdf},
- }
|
