Reinforcement Learning 2015/2016

Any slides of future lectures below are from previous year and will be updated about one day before each lecture. Lectures will be held between 12:10 - 13:00 in Teviot Lecture Theatre, Medical School, Doorway 5 on Tuesdays and same time same place on Fridays.
Basic Mathematical Background: Please review this cribsheet to make sure you understand the concepts therein. You may also find these resources useful as occasional reference material.
On Using Matlab: Take a look at this handout Introduction to MATLAB giving an introduction to MATLAB (you may ignore the section about NETLAB). A further MATLAB tutorial is available at MTU Introduction to Matlab.

Lecture topic (may change until link to slides is present):
January 12 2016
Slides (pdf)
Reading: Ch 1 of Sutton & Barto book
January 15 2016
Multi-Armed Bandits
Slides (pdf)
Reading: Ch 2 of Sutton & Barto book
January 19 2016
Q-learning I
Slides (pdf)
Reading: see last slide
January 21 2016
Homework 1 assigned
January 22 2016
Q-learning II
Slides (pdf)
Reading: See slides 14 and 15 for suggestions
January 26 2016
More RL algorithms
Slides (pdf)
Reading: Ch 6.4 and 11.2 of Sutton and Barto (2)
January 29 2016
Markov Decision Problems
Slides (pdf)
Reading: Ch. 3 of Sutton and Barto (2nd ed.) in particular 3.5 and 3.6
February 2 2016
The Bellman equation, eligibility traces
Slides (pdf)
Reading: see last slide for a reading suggestion
February 5 2016
Value iteration and policy iteration
Slides (pdf)
Reading: Ch. 4 of Sutton and Barto (2nd ed.) in particular 4.3 and 4.4
February 9 2016
State Abstraction
Slides (pdf)
Chapters 15 and 16 of S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics, MIT Press. Literature on last slide.
February 11 2016
Homework 1 due (Deadline: 16:00)
February 12 2016
RL with function approximation
Slides (pdf)

February 23 2016
RL with function approximation ctd.
Slides (pdf)
Literature; C. Szepesvari: Algorithms for RL, Chapter 2.2
February 25 2016
Homework 2 assigned
February 26 2016
Policy gradient methods, natural actor-critic
Slides (pdf)
Literature on last slide.
March 1 2016
Slides (pdf)
March 4 2016
Slides (pdf)
March 8 2016
Apprenticeship learning and inverse RL
Slides (pdf)
March 11 2016
Model-based and Multi-objective RL
Slides (pdf)
March 15 2016
Multi-Agent RL
Slides (pdf)
March 17 2016
Homework 2 due (Deadline: 16:00)
March 18 2016
Self-motivated RL.
Slides (pdf)
March 22 2016
Biological and neural RL (Bonus lecture, not examinable, JFYI)
Slides (pdf)
April 22 2016
Revision, questions, feedback (date and place to be confirmed)


The course includes 8 tutorials. Tutorials will start in week 3. Please contact the lecturer if you are not assigned to any group by week 2.

RL Home

Home : Teaching : Courses : Rl 

Informatics Forum, 10 Crichton Street, Edinburgh, EH8 9AB, Scotland, UK
Tel: +44 131 651 5661, Fax: +44 131 651 1426, E-mail:
Please contact our webadmin with any comments or corrections. Logging and Cookies
Unless explicitly stated otherwise, all material is copyright © The University of Edinburgh