Reinforcement Learning 2013/2014

Any slides of future lectures below are from previous year and will be updated about one day before each lecture.

Lectures will be held between 12:10 - 13:00 in LT4 7BSq. on Tuesdays and 12:10 - 13:00 same place on Fridays.
Basic Mathematical Background: Please review this cribsheet to make sure you understand the concepts therein. You may also find these resources useful as occasional reference material.
On Using Matlab: Take a look at this handout Introduction to MATLAB giving an introduction to MATLAB (you may ignore the section about NETLAB). A further MATLAB tutorial is available at MTU Introduction to Matlab.

Date:
Lecture topic :
January 14 2014
Introduction
Slides (pdf)
Reading: Ch 1 of Sutton & Barto book
January 17 2014
Multi-Armed Bandits
Slides (pdf)
Reading: Ch 2 of Sutton & Barto book
January 21 2014
A phenomenological account of Q-learning
Slides (pdf)
Reading: Ch 2 of Sutton & Barto book
January 24 2014
Reinforcement Learning: Examples; Markov Chains
Slides (pdf)
Reading: Ch 3 of Sutton & Barto book
Optional Reading: If you are curious about the Markov Chain portion and want to know more, there are many textbook type references, e.g., Ch 1 of a book by J. Norris.
January 28 2014
Markov decision problems
Slides (pdf)
Reading: Ch 1 of Szepesvari book
January 31 2014
Value functions and the Bellman equation, value prediction
Slides (pdf)
Reading: Ch 2.1 of Szepesvari book
February 4 2014
Reinforcement learning: Eligibility traces
Slides (pdf)
Reading: see slides for reading suggestions
February 6 2014
Homework 1 assigned
February 7 2014
Value iteration and policy iteration
Slides (pdf)
Reading: see slides for reading suggestions
February 11 2014
RL Algorithms and State Abstraction
Slides (pdf)
Literature on last slide.
February 14 2014
Hierarchical RL
Slides (pdf)

February 25 2014
POMDPs
Slides (pdf)
Literature on last slide.
February 28 2014
POMDPs continued
Slides (pdf)
Literature on last slide.
March 4 2014
RL with Function Approximation
Slides (pdf)
Based on C. Szepesvari: Algorithms for RL, Chapter 2.2
March 7 2014
Homework 1 due (Deadline: 16:00, moved from previously March 6, 16:00)
March 7 2014
Homework 2 assigned
March 7 2014
Policy gradient methods, natural actor-critic)
Slides (pdf)
March 11 2014
Complexity of RL
Slides (pdf)
March 14 2014
Complexity and convergence
Slides (pdf)
March 18 2014
Apprenticeship learning and inverse RL
Slides (pdf)
March 21 2014
no lecture

March 25 2014
A unfied view and recent trends (not examinable, JFYI)
Slides (pdf)
March 27 2014
Homework 2 due (Deadline: 16:00)
March 28 2014
no lecture

Bonus material
Biological RL (not examinable, JFYI)
Slides 1 (pdf), Slides 2 (pdf)

Tutorials

The course includes 8 tutorials. Tutorials will start in week 3. Please contact the lecturer if you are not assigned to any group by week 2.

RL Home


Home : Teaching : Courses : Rl 

Informatics Forum, 10 Crichton Street, Edinburgh, EH8 9AB, Scotland, UK
Tel: +44 131 651 5661, Fax: +44 131 651 1426, E-mail: school-office@inf.ed.ac.uk
Please contact our webadmin with any comments or corrections. Logging and Cookies
Unless explicitly stated otherwise, all material is copyright © The University of Edinburgh