RL 201718  Home  Lectures  Piazza  Github  Coursework  Tutorials
Basic Mathematical Background: Please review this cribsheet to make sure you understand the concepts therein. You may also find these resources useful as occasional reference material.
On Using Matlab: Take a look at this handout Introduction to MATLAB giving an introduction to MATLAB (you may ignore the section about NETLAB). A further MATLAB tutorial is available at MTU Introduction to Matlab.
Note that the coursework might also require other tools and programming environments. If so, then these will be introduced and explained in lectures.
Date: 
Lecture content: 
Assignments and Deadlines: 
January 16, 2018 
Introduction to Reinforcement Learning (RL) Slides (pdf) Reading: Ch 1 of Sutton & Barto book (1st ed.), and (optional) Section 17.2 of Kevin Murphy, Machine Learning: a Probabilistic Perspective (available online through University Library) 

January 19, 2018 
Introduction to Markov Decision Processes (MDPs) Slides (pdf) Reading: Ch 3 of Sutton & Barto book (1st ed.) 

January 23, 2018 
Introduction to Markov Decision Processes (continued); Example RL Problems from the Literature Slides (pdf) Examples (pdf) Reading: Ch 3 of Sutton & Barto book (1st ed.) 

January 26, 2018 
Introduction to the Matlab Base Code for the Course Assignments Code repository Slides (pdf) 

January 30, 2018 
Dynamic Programming for RL Slides (pdf) Reading: Ch 4 (up till 4.4) of Sutton & Barto book (1st ed.) 

February 2, 2018 
Dynamic Programming for RL (continued) Slides (pdf) Reading: Ch 4 of Sutton & Barto book (1st ed.) 

February 6, 2018 

Course Assignment 1 out (Code repository) 
February 9, 2018 
Discussion/Questions on Course Assignment 1 // Monte Carlo Methods for RL Slides (pdf) Reading: Ch 5 (up till 5.4) of Sutton & Barto book (1st ed.) 

February 13, 2018 
Monte Carlo Methods for RL (continued) // TemporalDifference Learning for RL Slides (pdf) Reading: Ch 5 (5.5 till end) and Ch 6 (6.1 to 6.3) of Sutton & Barto book (1st ed.) 

February 16, 2018 
TemporalDifference Learning for RL (continued) Slides (pdf) Reading: Ch 6 (6.4 to 6.5) of Sutton & Barto book (1st ed.) 

February 1923, 2018 
Flexible Learning Week 
Flexible Learning Week 
February 27, 2018 
[inclass Tutorial] Revision: RL Basics 

March 2, 2018 


March 5, 2018 

Assignment 1 Due (4 pm, submit electronically only) 
March 6, 2018 
Generalisation and Function Approximation Slides (pdf) Reading: Ch 8 of Sutton & Barto book (1st ed.) 
Course Assignment 2 out (Code repository) 
March 9, 2018 
Discussion/Questions on Course Assignment 2 

March 13, 2018 
Eligibility Traces Slides (pdf) Reading: Ch 7 of Sutton & Barto book (1st ed.) 

March 16, 2018 
[inclass Tutorial] Exercise on RL Basics Exercise. Solution. 

March 20, 2018 
Abstraction: Options and Hierarchies Slides (pdf) Reading: Case study, Sec 11.4 (Elevator Dispatching) in print version of S+B book, and up to and including Sec 4.1 from A.G. Barto, S. Mahadevan, Recent Advances in Hierarchical Reinforcement Learning, Discrete Event Dynamic Systems 13(4):341379, 2003. You can get the article via SpringerLink or get the preprint version here. Optional Reading: R.S. Sutton, D. Precup, S. Singh, Between MDPs and semiMDPs: A framework for temporal abstraction in reinforcement learning, Artificial Intelligence, Vol. 112, pp. 181  211, 1999. (ElsevierLink) 

March 23, 2018 
Partially Observable MDPs Slides (pdf) (based on material associated with Thrun et al. book) Optional Reading: Chapter 15 of S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics, MIT Press. 

March 27, 2018 
Inverse Reinforcement Learning Slides (pdf) Reading: Up to and including Section 4 of A.Y. Ng, S.J. Russell, Algorithms for inverse reinforcement learning. In Proc. ICML, pp. 663670, 2000. (Preprint here ). 

March 28, 2018 (same time and place) 
[inclass Tutorial] Exercise on Function Approximation Exercise. Solution. 

March 30, 2018 
Multiagent Reinforcement Learning Slides (pdf) Reading: M. Bowling, M. Veloso, An analysis of stochastic game theory for multiagent reinforcement learning, CMU Technical Report CMUCS00165, 2000. 

April 2, 2018 

Assignment 2 Due (4 pm, submit electronically only) 
April 3, 2018 
Multiagent Reinforcement Learning (cont.): minimax Search // Exam checklist Slides (pdf) Exam "To Know" List (pdf) 

April 6, 2018 
[inclass Tutorial] Exam Mockup and Q+A 
Informatics Forum, 10 Crichton Street, Edinburgh, EH8 9AB, Scotland, UK
Tel: +44 131 651 5661, Fax: +44 131 651 1426, Email: schooloffice@inf.ed.ac.uk Please contact our webadmin with any comments or corrections. Logging and Cookies Unless explicitly stated otherwise, all material is copyright © The University of Edinburgh 