Reinforcement Learning 2015/2016

Any slides of future lectures below are from previous year and will be updated about one day before each lecture. Lectures will be held between 12:10 - 13:00 in Teviot Lecture Theatre, Medical School, Doorway 5 on Tuesdays and same time same place on Fridays.
Basic Mathematical Background: Please review this cribsheet to make sure you understand the concepts therein. You may also find these resources useful as occasional reference material.
On Using Matlab: Take a look at this handout Introduction to MATLAB giving an introduction to MATLAB (you may ignore the section about NETLAB). A further MATLAB tutorial is available at MTU Introduction to Matlab.

Lecture topic (may change until link to slides is present):
January 12 2016
Slides (pdf)
Reading: Ch 1 of Sutton & Barto book
January 15 2016
Multi-Armed Bandits
Slides (pdf)
Reading: Ch 2 of Sutton & Barto book
January 19 2016
Q-learning I
Slides (pdf)
Reading: see last slide
January 21 2016
Homework 1 assigned
January 22 2016
Q-learning II
Slides (pdf)
Reading: See slides 14 and 15 for suggestions
January 26 2016
More RL algorithms
Slides (pdf)
Reading: Ch 6.4 and 11.2 of Sutton and Barto (2)
January 29 2016
Markov Decision Problems
Slides (pdf)
Reading: Ch. 3 of Sutton and Barto (2nd ed.) in particular 3.5 and 3.6
February 2 2016
The Bellman equation, eligibility traces
Slides (pdf)
Reading: see last slide for a reading suggestion
February 5 2016
Value iteration and policy iteration
Slides (pdf)
Reading: Ch. 4 of Sutton and Barto (2nd ed.) in particular 4.3 and 4.4
February 9 2016
State Abstraction
Slides (pdf)
Chapters 15 and 16 of S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics, MIT Press. Literature on last slide.
February 11 2016
Homework 1 due (Deadline: 16:00)
February 12 2016
RL with function approximation
Slides (pdf)

February 23 2016
RL with function approximation ctd.
Slides (pdf) (suppl.)
Literature; C. Szepesvari: Algorithms for RL, Chapter 2.2
February 25 2016
Homework 2 assigned Homework 2 assigned
February 26 2016
Slides (pdf)
Literature on last slide.
March 1 2016
Slides (pdf) (suppl.)
March 4 2016
POMDPs IIa and Model-based RL
Slides (pdf)
March 8 2016
Apprenticeship learning and inverse RL
Slides (pdf)
March 11 2016
Multi-objective RL
Slides (pdf)
March 15 2016
Multi-Agent RL and Deep RL
Slides (pdf) and (pdf)
March 17 2016
Homework 2 due (Deadline: 16:00)
March 18 2016
Self-motivated RL and Biological RL
Slides (pdf1) and (pdf2)
April 22 2016
Revision, questions, feedback (date and place to be confirmed)


The course includes 8 tutorials. Tutorials will start in week 3. Please contact the lecturer if you are not assigned to any group by week 2.

RL Home

Home : Teaching : Courses : Rl 

Informatics Forum, 10 Crichton Street, Edinburgh, EH8 9AB, Scotland, UK
Tel: +44 131 651 5661, Fax: +44 131 651 1426, E-mail:
Please contact our webadmin with any comments or corrections. Logging and Cookies
Unless explicitly stated otherwise, all material is copyright © The University of Edinburgh