University Homepage School Homepage School Contacts School Search

Reinforcement Learning

Lectures (to be updated before each lecture)

Lectures will be held between 12:10 - 13:00 in LT4 7BSq. on Tuesdays and 12:10 - 13:00 same place on Fridays.

Basic Mathematical Background: Please review this cribsheet to make sure you understand the concepts therein. You may also find these resources useful as occasional reference material.

On Using Matlab: Take a look at this handout Introduction to MATLAB giving an introduction to MATLAB (you may ignore the section about NETLAB). Further MATLAB tutorials are available at UNH Matlab Tutorial, US Navy Matlab Tutorial and MTU Introduction to Matlab.

Date:	Lecture content:
January 15	Introduction Slides (pdf) Reading: Ch 1 of Sutton & Barto book
January 18	Multi-Armed Bandits Slides (pdf) Reading: Ch 2 of Sutton & Barto book
January 22	A phenomenological account of Q-learning Slides (pdf) Reading: Ch 2 of Sutton & Barto book
January 25	Formulation of the Reinforcement Learning problem - Markov Decision Processes Slides (pdf) Reading: Ch 3 of Sutton & Barto book Optional Reading: If you are curious about the Markov Chain portion and want to know more, there are many textbook type references, e.g., Ch 1 of a book by J. Norris.
January 29	Value functions and the Bellman equation Slides (pdf) Reading: Ch 1 of Szepesvari book
February 1	Value Prediction Slides (pdf) Reading: Ch 2.1 of Szepesvari book
February 5	Reinforcement learning: Examples Learning Slides (pdf) Reading: see slides for reading suggestions
February 8	Biological Reinforcement learning Slides (pdf) Reading: see slides for reading suggestions	Homework 1 assigned
February 12	POMDP Slides (pdf) Chapters 15 and 16 of S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics, MIT Press.
February 15	POMDP Slides (pdf)
February 26	Hierarchical RL 1 Slides (pdf) Literature on last slide.
March 1	Hierarchical RL 2 Slides (pdf) Literature on last slide.
March 5	Large State Spaces Slides (pdf) Based on C. Szepesvari: Algorithms for RL, Chapter 2.2
March 7	Deadline has been extended by one week (because of ILW)	Homework 1 due (Deadline: 16:00)
March 8	Policy gradient methods (Actor-critic) Slides (pdf)	Homework 2 assigned
March 12	Natural Actor Critic Slides (pdf)
March 15	NO LECTURE
March 19	Particle swarms and Further topics in RL Slides (pdf)
March 22	Efficient representations for RL Slides (pdf)
March 26	RL in neural networks (bonus lecture) Slides (pdf)
March 28		Homework 2 due (Deadline: 16:00)

Tutorials

The course does now include tutorials. Tutorials are on Mondays and Thursdays, 1pm in AT 5.07. Please contact the lecturer if you want to join one of the groups.

• Week of 4th Feb: Bandits and MDP formulation - Questions (with hints)
• Week of 11th Feb: Dynamic programming - Questions (with hints)
• Week of 25th Feb: Applications and POMDPs Questions (no hints)
• Week of 4th Mar: Theory and applications Questions (no hints yet)
• Week of 11th Mar: POMDPs - Questions
• Week of 18th Mar: Revision

RL Home