Neural Information Processing: Course homepage 2011-2012

This is a course for MSc level students. It runs in semester 2. The course descriptor can be found here.

This course builds on recent insights that in many cases the computation done by the nervous system can be described using techniques from probabilistic modelling (machine learning) and information theory. The aim will be to examine work on computation in nervous system in terms of these methods.

The background needed to successfully take this course is a good grounding in mathematics, particularly with regard to probability and statistics, vectors and matrices. The mathematical level required is similar to that which would be obtained by students who did not have significant difficulties with the courses Mathematics for Informatics 1-4 taken in the first two years of the Informatics undergraduate syllabus. The Probabilistic Modelling & Reasoning (PMR) course is strongly recommended as preparation. If in doubt on this consult the instructors.

Instructors are Mark van Rossum (first half) and Chris Williams (second half).
Course Tutor: Dagmara Panas

Venue

Monday and Thursday 9.00-9.50, AT.M3 (AT mezzanine level, same level as AT lecture theatres 4 and 5).
First lecture Monday 16 Januari.

Course Outline

Overview of relevant neurobiology
Neural encoding
Neural decoding
Mathematics of Hebbian learning
Models of early visual coding (sparse coding, ICA and beyond)
Lateral Interactions and Feedback
Object recognition

Assessment

There will be two assessed assignments worth in total 25%. There will be an exam worth 75%.

The first assignment is now available, with a deadline of 5 Mar at 4pm. We remind students that late submissions are not allowed.

The second assignment is now available, with a deadline of Fri 30 Mar at 4pm. We remind students that late submissions are not allowed. Associated files: house.mat, nip_question_2.mat, PlotFilterInSpace.m, creategratings.m, display_network_nonsquare2.m

Books

Theoretical Neuroscience by P Dayan and L F Abbott (MIT Press 2001) is recommended reading see also the list of errata.
Natural Image Statistics by Aapo Hyv�rinen, Jarmo Hurri, and Patrik O. Hoyer. Full version online. This will be supplemented by papers from the literature.

Week-by-week listing

Week 1 (16/1)

Lectures: Introduction: lecture slides (2x2) , lecture slides (single page)
Neural encoding: lecture slides (2x2) , lecture slides (single page)

Week 2 (23/1)

Lectures: Neural encoding (cont.)

Week 3 (30/1)

Neural decoding: lecture slides, lecture slides (single page)

Week 4 (6/2)

Lectures: Information theory: lecture slides, lecture slides (single page)

Week 5 (13/2)

Lectures: Information theory (cont.), Predicting Retinal Ganglion Cell Receptive Fields, slides (single page), slides (4up)
Web resource: Contrast sensitivity section from Visual Acuity chapter from Psychophysics of Vision (Michael Kalloniatis and Charles Luu) at http://webvision.med.utah.edu/
Background reading on Fourier analysis: Fourier series, Fourier transform

Week 6 (20/2)

No Lectures. Innovative learning week.

Week 7 (27/2)

Lectures: Higher Order Statistics slides (single page), slides (4up),
Beyond ICA slides (single page), slides (4up)
References
Emergence of Simple-Cell Receptive Field Properties by Learning a Sparse Code for Natural Images, Olshausen BA, Field DJ Nature, 381: 607-609 (1996)
Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1? Olshausen BA, Field DJ, Vision Research, 37: 3311-3325 (1997).
The "independent components" of natural scenes are edge filters. Anthony J. Bell and Terrence J. Sejnowski, Vision Research 37(23): 3327-3338 (1997). preprint, official paper also available electronically via UoE library (Science Direct)
Further References (details not required for NIP course)
A hierarchical Bayesian model for learning non-linear statistical regularities in non-stationary natural signals, Y. Karklin and M. S. Lewicki, Neural Computation, 17 (2): 397-423, (2005)
Learning higher-order structures in natural images, Y. Karklin and M. S. Lewicki, Network: Computation in Neural Systems, 14: 483-499, (2003)
Topographic Independent Component Analysis, A. Hyvarinen, P.O. Hoyer and M. Inki, Neural Computation, 13(7):1527-1558, (2001)
Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces A. Hyvarinen and P.O. Hoyer, Neural Computation, 12(7):1705-1720, (2000)

Week 8 (5/3)

Lectures: Beyond ICA continued, Undirected Graphical Models lecture slides (single page), lecture slides (4up)
References
Sparse deep belief net model for visual area V2. Honglak Lee, Ekanadham Chaitanya, and Andrew Y. Ng. Advances in Neural Information Processing Systems 20 (2008).
Topographic Product Models Applied To Natural Scene Statistics Osindero, S., Welling, M. and Hinton, G. E. Neural Computation, 18(2) (2006).

Week 9 (12/3)

Lectures: Evaluating Models of Natural Image Patches lecture slides (single page), lecture slides (4up)
Lateral Interactions and Feedback lecture slides (single page), lecture slides (4up)
References
Natural Image Coding in V1: How Much Use is Orientation Selectivity? Eichhorn J, Sinz FH and Bethge M, PLoS Computational Biology 5(4:e1000336) 1-16 (2009)
Predictive Coding in the Visual Cortex. Rao, R. P. N. and Ballard, D. H., Nature Neuroscience, 2(1), 79-87, (1999)
Hierarchical Bayesian inference in the visual cortex. Lee, T.S., Mumford, D., Journal of Optical Society of America, A. . 20(7): 1434-1448 (2003)

Week 10 (19/3)

Lectures: Object recognition: lecture slides (single page) lecture slides (4up)
Reading Do we know what the early visual system does?, M. Carandini, J. B. Demb, V. Mante, D. J. Tolhurst, Y. Dang, B.A. Olshausen, J. L. Gallant, N. C. Rust, J Neurosci 25: 10577-10597 (2005)
Further References (details not required for NIP course)
Hierarchical models of object recognition in cortex. Maximilian Riesenhuber & Tomaso Poggio. Nature Neuroscience 2(11) (1999)
Robust Object Recognition with Cortex-Like Mechanisms Serre, T., L. Wolf, S. Bilschi, M. Reisenhuber and T. Poggio. IEEE PAMI 29(3) 411-426 (2007)

This page is maintained by Chris Williams and Mark van Rossum .

Home : Teaching : Courses