Reinforcement Learning 2014/2015

Lectures will be held between 12:10 - 13:00 in AT LT2 on Tuesdays and 12:10 - 13:00 same place on Fridays.
Basic Mathematical Background: Please review this cribsheet to make sure you understand the concepts therein. You may also find these resources useful as occasional reference material.
On Using Matlab: Take a look at this handout Introduction to MATLAB giving an introduction to MATLAB (you may ignore the section about NETLAB). A further MATLAB tutorial is available at MTU Introduction to Matlab.

Lecture topic (may change until link to slides is present):
January 13 2015
Slides (pdf)
Reading: Ch 1 of Sutton & Barto book
January 16 2015
Multi-Armed Bandits
Slides (pdf)
Reading: Ch 2 of Sutton & Barto book
January 20 2015
Q-learning I
Slides (pdf)
Reading: see last slide
January 23 2015
Q-learning II
Slides (pdf)
Reading: See slides 14 and 15 for suggestions
January 27 2015
More RL algorithms
Slides (pdf)
Reading: Ch 6.4 and 11.2 of Sutton and Barto (2)
January 30 2015
Markov Decision Problems
Slides (pdf)
Reading: Ch. 3 of Sutton and Barto (2nd ed.) in particular 3.5 and 3.6
February 3 2015
The Bellman equation, eligibility traces
Slides (pdf)
Reading: see last slide for a reading suggestion
February 5 2015
Homework 1 assigned
February 6 2015
Value iteration and policy iteration
Slides (pdf)
Reading: Ch. 4 of Sutton and Barto (2nd ed.) in particular 4.3 and 4.4
February 10 2015
State Abstraction
Slides (pdf)
Chapters 15 and 16 of S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics, MIT Press. Literature on last slide.
February 13 2015
RL with function approximation
Slides (pdf)

February 24 2015
RL with function approximation ctd.
Slides (pdf)
Literature; C. Szepesvari: Algorithms for RL, Chapter 2.2
February 27 2015
Policy gradient methods, natural actor-critic
Slides (pdf)
Literature on last slide.
March 3 2015
Slides (pdf)
March 5 2015
Homework 1 due (Deadline: 16:00)
March 5 2015
Homework 2 assigned
March 6 2015
Slides (pdf)
March 10 2015
Apprenticeship learning and inverse RL
Slides (pdf)
March 13 2015
Model-based and Multi-objective RL
Slides (pdf)
March 17 2015
Multi-Agent RL
Slides (pdf)
March 20 2015
Self-motivated RL.
Slides (pdf)
March 24 2015
Biological and neural RL (Bonus lecture, not examinable, JFYI)
Slides (pdf)
March 26 2015
Homework 2 due (Deadline: 16:00)
April 24 2015
Revision, questions, feedback (date and place to be confirmed)


The course includes 8 tutorials. Tutorials will start in week 3. Please contact the lecturer if you are not assigned to any group by week 2.

RL Home

Home : Teaching : Courses : Rl 

Informatics Forum, 10 Crichton Street, Edinburgh, EH8 9AB, Scotland, UK
Tel: +44 131 651 5661, Fax: +44 131 651 1426, E-mail:
Please contact our webadmin with any comments or corrections. Logging and Cookies
Unless explicitly stated otherwise, all material is copyright © The University of Edinburgh