Machine Learning Practical (MLP) 2016-17

[course descriptor]

News

Introduction

The coursework-based Machine Learning Practical (MLP) is focused on the implementation and evaluation of machine learning systems. Students who do this course will have experience in the design, implementation, training, and evaluation of machine learning systems.

In 2016-17 the course will focus on deep neural networks. There are some changes compared with the 2015-16 MLP course, most notably the course is now 20 credits, and runs through both semesters.

MLP requires mathematical ability (calculus, linear algebra, and probability) and programming ability (the course will be based on Python using Numpy). Some previous experience of machine learning is is extremely helpful.
Undergraduates: If you have taken Informatics 2B and IAML, and can program, you are qualified to do the course.
It is also recommended to take MLPR.

MLP 2016-17: Deep Neural Networks

This year the MLP course will focus on deep neural networks. During semester 1 we shall focus on the classification of handwritten digits using the well-known MNIST dataset. Using a Python software framework that we shall provide, and a series of iPython notebooks, the aim of the course is to train multi-layer neural neural network classifiers and convolutional network classifiers to address this handwritten digit classification problem. There will be a series of nine weekly lectures in semester 1 to provide the required theoretical support to the practical work.

In semester 2, we shall focus on using deep neural networks to address a second task. In semester 2 there will be a series of five weekly lectures (weeks 1-5) to support the coursework, and also to provide insights to the current state of the art in this very fast moving area.

Schedule

Lectures

Semester 1 (9 lectures)

All lectures at 11am on Wednesdays
Lectures in Basement Lecture Theatre, Adam House, 3 Chambers Street.
Lecture recordings.

  1. Wednesday 21 September 2016 (Informatics Forum, G.07/G.07A). Introduction to MLP. Linear networks. Gradient descent.
    Slides.
    Lecture recordings: Part 1 - Introduction;   Part 2 - Single Layer Networks.
     
  2. Wednesday 28 September 2016. Stochastic gradient descent. Classification.
    Slides.
    Lecture recordings: Part 1 - Stochastic Gradient Descent;   Part 2 - Classification, Sigmoid, and Softmax.
     
  3. Wednesday 5 October 2016. Multi-layer networks.
    Slides.
    Lecture recordings: Part 1 - Classification with Neural Networks - Templates and Features;   Part 2 - Learning Multi-layer Networks.
     
  4. Wednesday 12 October 2016. Adaptive learning rates. First coursework. Generalisation.
    Slides.
    Lecture recordings: Part 1 - Learning rates and momentum; coursework 1;   Part 2 - Generalisation in practice.
     
  5. Wednesday 19 October 2016. Regularisation. Hidden Unit Transfer Functions.
    Slides.
    Lecture recordings: Part 1 - Regularisation;   Part 2 - Hidden unit transfer functions.
     
  6. Wednesday 26 October 2016. Normalisation. Initialisation. Pretraining.
    Slides.
    Lecture recordings: Part 1 - Feature normalisation, random initialisation, batch normalisation;   Part 2 - Stacked pretraining.
     
  7. Wednesday 2 November 2016. Introduction to coursework 2; Convolutional networks (1).
    Slides.
    Lecture recordings: Part 1 - Introduction to coursework 2;   Part 2 - Convolutional neural networks.
     
  8. Wednesday 9 November 2016. Convolutional networks (2).
    Slides.
    Lecture recordings: Part 1 - Gradient descent training of convolutional networks;   Part 2 - Recent advances usng convolutional networks.
     
  9. Wednesday 16 November 2016. Recurrent neural networks.
    Slides.
    Lecture recordings: Part 1 - Introduction to recurrent neural networks, backprop through time, LSTM.;   Part 2 - Applications of RNNs.

Semester 2 (5 lectures)

All lectures at 11am on Wednesdays, Appleton Tower LT1.

  1. Wednesday 18 January 2017. Introduction to part 2 of MLP.
    Slides.
    Lecture recordings: MLP in Semester 2.
  2. Wednesday 25 January 2017. Multi-task learning and related supervised methods. Peter Bell.
    Slides.
    Lecture recordings: Multi-task learning and related supervised methods.
  3. Wednesday 1 February 2017. Discussion lecture.
  4. Wednesday 8 February 2017. Benigno Uria (DeepMind).
  5. Wednesday 15 February 2017. Ben Allison (Amazon), "Building Production Machine Learning Systems"

You can discuss and ask questions about these lectures on the online MLP Forum.

Labs

The lab material is available using GitHub at https://github.com/CSTR-Edinburgh/mlpractical. You do not require a github login to use this. Each new lab corresponds to a new branch in git: see Getting started in a Lab for details about this.

Labs use iPython and Jupyter notebook. There are many Python/numpy tutorials on the web, I think that this is a good one: http://cs231n.github.io/python-numpy-tutorial/.

Solutions for each week's lab notebook will be included in the corresponding notebook in the branch you checkout with the following week's material.

Lab Sessions

You should attend one 1-hour scheduled lab session per week.
Semester 2: All labs will take place in Forrest Hill, room 3.D01. Labs will take place on Mondays (9-10am; 10-11am), Tuesdays (1-2pm; 2-3pm), and Thursdays (9-10am; 10-11am).
Semester 1: All labs will take place in Forrest Hill, room 3.D01. Labs will take place on Mondays (1-2pm; 2-3pm), Tuesdays (1-2pm; 2-3pm), and Thursdays (1-2pm; 2-3pm).

Office hours

MLP Office Hour from 4-5pm on Fridays, 3.15 in the Informatics Forum, (7 October - 18 November) If you have a question about the course, then you are welcome to come along and ask/discuss. If your question is about the software, the environment, or implementation issues relating to the labs or coursework, then please ask your questions in the lab, rather than the office hour.

Coursework

Please make sure you have read and understood

Reading

Textbooks

Additional material

Review articles (including material not covered in this course)

Course Feedback

We are very keen to hear any feedback on MLP - what are we doing right? what could be done better? what might we consider introducing? Contact me any time if you have any comments.

There is also an official feedback form near the end of each semester, please do take the time to complete it. Mid-semester, there is also a briefer feedback opportunity.


This page maintained by Steve Renals.
Last updated: 2017/04/14 10:49:44UTC


Home : Teaching : Courses 

Informatics Forum, 10 Crichton Street, Edinburgh, EH8 9AB, Scotland, UK
Tel: +44 131 651 5661, Fax: +44 131 651 1426, E-mail: school-office@inf.ed.ac.uk
Please contact our webadmin with any comments or corrections. Logging and Cookies
Unless explicitly stated otherwise, all material is copyright © The University of Edinburgh