Reinforcement Learning 2017-2018

Typically, lecture slides will be added/updated one day before the lecture. Lectures will be held between 12:10 - 13:00 in Teviot Lecture Theatre, Medical School, Doorway 5 on Tuesdays and same time same place on Fridays.

Basic Mathematical Background: Please review this cribsheet to make sure you understand the concepts therein. You may also find these resources useful as occasional reference material.

On Using Matlab: Take a look at this handout Introduction to MATLAB giving an introduction to MATLAB (you may ignore the section about NETLAB). A further MATLAB tutorial is available at MTU Introduction to Matlab.
Note that the coursework might also require other tools and programming environments. If so, then these will be introduced and explained in lectures.

Date:	Lecture content:	Assignments and Deadlines:
January 16, 2018	Introduction to Reinforcement Learning (RL) Slides (pdf) Reading: Ch 1 of Sutton & Barto book (1st ed.), and (optional) Section 17.2 of Kevin Murphy, Machine Learning: a Probabilistic Perspective (available online through University Library)
January 19, 2018	Introduction to Markov Decision Processes (MDPs) Slides (pdf) Reading: Ch 3 of Sutton & Barto book (1st ed.)
January 23, 2018	Introduction to Markov Decision Processes (continued); Example RL Problems from the Literature Slides (pdf) Examples (pdf) Reading: Ch 3 of Sutton & Barto book (1st ed.)
January 26, 2018	Introduction to the Matlab Base Code for the Course Assignments Code repository Slides (pdf)
January 30, 2018	Dynamic Programming for RL Slides (pdf) Reading: Ch 4 (up till 4.4) of Sutton & Barto book (1st ed.)
February 2, 2018	Dynamic Programming for RL (continued) Slides (pdf) Reading: Ch 4 of Sutton & Barto book (1st ed.)
February 6, 2018		Course Assignment 1 out (Code repository)
February 9, 2018	Discussion/Questions on Course Assignment 1 // Monte Carlo Methods for RL Slides (pdf) Reading: Ch 5 (up till 5.4) of Sutton & Barto book (1st ed.)
February 13, 2018	Monte Carlo Methods for RL (continued) // Temporal-Difference Learning for RL Slides (pdf) Reading: Ch 5 (5.5 till end) and Ch 6 (6.1 to 6.3) of Sutton & Barto book (1st ed.)
February 16, 2018	Temporal-Difference Learning for RL (continued) Slides (pdf) Reading: Ch 6 (6.4 to 6.5) of Sutton & Barto book (1st ed.)
February 19-23, 2018	Flexible Learning Week	Flexible Learning Week
February 27, 2018	[in-class Tutorial] Revision: RL Basics
March 2, 2018
March 5, 2018		Assignment 1 Due (4 pm, submit electronically only)
March 6, 2018	Generalisation and Function Approximation Slides (pdf) Reading: Ch 8 of Sutton & Barto book (1st ed.)	Course Assignment 2 out (Code repository)
March 9, 2018	Discussion/Questions on Course Assignment 2
March 13, 2018	Eligibility Traces Slides (pdf) Reading: Ch 7 of Sutton & Barto book (1st ed.)
March 16, 2018	[in-class Tutorial] Exercise on RL Basics Exercise. Solution.
March 20, 2018	Abstraction: Options and Hierarchies Slides (pdf) Reading: Case study, Sec 11.4 (Elevator Dispatching) in print version of S+B book, and up to and including Sec 4.1 from A.G. Barto, S. Mahadevan, Recent Advances in Hierarchical Reinforcement Learning, Discrete Event Dynamic Systems 13(4):341-379, 2003. You can get the article via SpringerLink or get the preprint version here. Optional Reading: R.S. Sutton, D. Precup, S. Singh, Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning, Artificial Intelligence, Vol. 112, pp. 181 - 211, 1999. (ElsevierLink)
March 23, 2018	Partially Observable MDPs Slides (pdf) (based on material associated with Thrun et al. book) Optional Reading: Chapter 15 of S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics, MIT Press.
March 27, 2018	Inverse Reinforcement Learning Slides (pdf) Reading: Up to and including Section 4 of A.Y. Ng, S.J. Russell, Algorithms for inverse reinforcement learning. In Proc. ICML, pp. 663-670, 2000. (Preprint here ).
March 28, 2018 (same time and place)	[in-class Tutorial] Exercise on Function Approximation Exercise. Solution.
March 30, 2018	Multi-agent Reinforcement Learning Slides (pdf) Reading: M. Bowling, M. Veloso, An analysis of stochastic game theory for multiagent reinforcement learning, CMU Technical Report CMU-CS-00-165, 2000.
April 2, 2018		Assignment 2 Due (4 pm, submit electronically only)
April 3, 2018	Multi-agent Reinforcement Learning (cont.): minimax Search // Exam checklist Slides (pdf) Exam "To Know" List (pdf)
April 6, 2018	[in-class Tutorial] Exam Mock-up and Q+A

This page is maintained by the course lecturer

RL Home

Home : Teaching : Courses : Rl