# Decision Making in Robots and Autonomous Agents

### Semester 2 2018/2019

### Admin:

MSc Course organiser:

Dr Paul Jackson
For administrative queries, your first point of contact is the ITO. Also see

this link.

### Lectures:

Lecturer:

Dr Subramanian Ramamoorthy
Lecture times: Fridays 11:10 - 12:00 and 12:10 - 13:00

Location: Room G.3, Bayes Centre

Lecture topics and handouts
### Assessment:

The course mark will be computed using the following weighting:

- Two coursework assignments - 10% and 20% respectively
- Term paper (third coursework assignment) - 10%
- Final exam - 60%

### Primary Objectives

This course is intended as a specialized course on models and techniques for decision making (under uncertainty) in robots. The emphasis is on understanding how we can endow robots with the capacity to

*autonomously* make decisions about how to interact with a dynamic environment (including, sometimes, other agents in these environments). We will be motivated by questions such as the following:

- What kinds of decisions do robots need to make in realistic autonomy domains?
- What kinds of mathematical models and computational methods can be used to program such decision making behaviour?
- What issues must we consider, as designers of systems that have the capacity to be deployed as autonomous agents in their own right?

### Background and Pre-requisites

This is a 'second course' at the postgraduate level. A student following this course should have had some prior exposure to robotics and machine learning. For robotics, this could be through our introductory MSc level semester 1 course, R:SS. For machine learning, we assume knowledge of material covered in an introductory course such as IAML.

The student should be proficient in the formulation and use of mathematical models; and possess sufficient mathematical maturity to be able to follow some readings from the research literature. So, for instance, students ought to have working knowledge of multivariate calculus (e.g., Jacobian), probability (expectation, conditional probability), stochastic processes (Markov chains) and optimization (e.g., linear programming, gradient descent algorithms).

On the practical side, one of the assignments will require programming in a high level language such as Python. Students are expected to enter this course with sufficient programming skill, or the capacity to learn what is required on the fly. However, this is not a 'programming course' - the classroom discussion will focus on algorithmic and conceptual ideas.

### Suggested Readings

There is no single required text book for this course. In addition to lecture slides (made available through link above), the lecturer will assign readings from books and research articles.

Beyond these core materials, the curious student may find some of the following readings to be of interest:

- B. Christian, T. Griffiths, Algorithms to Live By, William Collins Press, 2016.
- R.D. Luce, H. Raiffa, Games and Decisions, Dover Publications, 1989.
- D.P. Bertsekas, Dynamic Programming and Optimal Control, Vols. I & II, Athena Press, 2017.
- M. Mitzenmacher, E. Upfal, Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis, Cambridge University Press, 2017.

**Last update:** 10 January 2019.