Neural Information Processing: Course homepage 2011-2012
This is a course for MSc level students. It runs in semester 2.
The course descriptor can
be found
here.
This course builds on recent insights that in many cases the
computation done by the nervous system can be described using
techniques from probabilistic modelling (machine
learning) and information theory. The aim will be to examine
work on computation in nervous system in terms of these
methods.
The background needed to successfully take this course is a
good grounding in mathematics, particularly with regard to probability
and statistics, vectors and matrices. The mathematical level required
is similar to that which would be obtained by students who did not
have significant difficulties with the courses Mathematics for
Informatics 1-4 taken in the first two years of the Informatics
undergraduate syllabus. The Probabilistic Modelling & Reasoning (PMR)
course is strongly recommended as preparation. If in doubt on this
consult the instructors.
Instructors are
Mark van Rossum (first half) and
Chris Williams (second half).
Course Tutor: Dagmara Panas
Venue
Monday and Thursday 9.00-9.50, AT.M3 (AT mezzanine level, same level
as AT lecture theatres 4 and 5).
First lecture Monday 16 Januari.
Course Outline
- Overview of relevant neurobiology
- Neural encoding
- Neural decoding
- Mathematics of Hebbian learning
- Models of early visual coding (sparse coding, ICA and beyond)
- Lateral Interactions and Feedback
- Object recognition
Assessment
There will be two assessed assignments worth in total 25%. There will
be an exam worth 75%.
The first assignment is now available, with a
deadline of 5 Mar at 4pm. We remind students that
late submissions are not allowed.
The second assignment is now available, with a
deadline of Fri 30 Mar at 4pm. We remind students that
late submissions are not allowed. Associated files:
house.mat,
nip_question_2.mat,
PlotFilterInSpace.m,
creategratings.m,
display_network_nonsquare2.m
Books
Theoretical Neuroscience
by P Dayan and L F Abbott (MIT Press 2001) is recommended reading see also the
list of
errata.
Natural Image Statistics by Aapo Hyvärinen, Jarmo Hurri, and Patrik O. Hoyer.
Full version online.
This will be supplemented by papers from the literature.
Week-by-week listing
Week 1 (16/1)
Lectures:
Introduction:
lecture slides (2x2) ,
lecture slides (single page)
Neural encoding:
lecture slides (2x2) ,
lecture slides (single page)
Week 2 (23/1)
Lectures:
Neural encoding (cont.)
Week 3 (30/1)
Neural decoding:
lecture slides,
lecture slides (single page)
Week 4 (6/2)
Lectures:
Information theory:
lecture slides,
lecture slides (single page)
Week 5 (13/2)
Lectures:
Information theory (cont.), Predicting Retinal Ganglion Cell Receptive Fields,
slides (single page),
slides (4up)
Web
resource:
Contrast
sensitivity section from Visual Acuity chapter from
Psychophysics of Vision (Michael Kalloniatis and Charles Luu)
at
http://webvision.med.utah.edu/
Background reading on Fourier analysis:
Fourier series,
Fourier transform
Week 6 (20/2)
No Lectures. Innovative learning week.
Week 7 (27/2)
Lectures: Higher Order Statistics
slides (single page),
slides (4up),
Beyond ICA
slides (single page),
slides (4up)
References
Emergence of Simple-Cell Receptive Field Properties by Learning a
Sparse Code for Natural Images, Olshausen BA, Field DJ Nature, 381:
607-609 (1996)
Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1?
Olshausen BA, Field DJ, Vision Research, 37: 3311-3325 (1997).
The "independent components" of natural scenes are edge filters.
Anthony J. Bell and Terrence J. Sejnowski, Vision Research 37(23): 3327-3338 (1997).
preprint,
official paper also available electronically via UoE library (Science
Direct)
Further References (
details not required for NIP course)
A hierarchical Bayesian model for learning non-linear statistical
regularities in non-stationary natural signals, Y. Karklin and
M. S. Lewicki, Neural Computation, 17 (2): 397-423, (2005)
Learning higher-order structures in natural images,
Y. Karklin and M. S. Lewicki,
Network: Computation in Neural Systems, 14: 483-499, (2003)
Topographic Independent Component Analysis,
A. Hyvarinen, P.O. Hoyer and M. Inki,
Neural Computation, 13(7):1527-1558, (2001)
Emergence of phase and shift invariant
features by decomposition of natural images into independent feature
subspaces A. Hyvarinen and P.O. Hoyer,
Neural Computation, 12(7):1705-1720, (2000)
Week 8 (5/3)
Lectures: Beyond ICA continued, Undirected Graphical Models
lecture slides (single page),
lecture slides (4up)
References
Sparse deep belief net model for visual area V2. Honglak Lee,
Ekanadham Chaitanya, and Andrew Y. Ng.
Advances in Neural Information Processing Systems 20 (2008).
Topographic Product Models Applied To Natural Scene Statistics
Osindero, S., Welling, M. and Hinton, G. E.
Neural Computation, 18(2) (2006).
Week 9 (12/3)
Lectures:
Evaluating Models of Natural Image Patches
lecture slides (single page),
lecture slides (4up)
Lateral Interactions and Feedback
lecture slides (single page),
lecture slides (4up)
References
Natural Image Coding in V1: How Much Use is Orientation Selectivity?
Eichhorn J, Sinz FH and Bethge M,
PLoS Computational Biology 5(4:e1000336) 1-16 (2009)
Predictive Coding in the Visual Cortex.
Rao, R. P. N. and Ballard, D. H.,
Nature Neuroscience, 2(1), 79-87, (1999)
Hierarchical Bayesian inference in the visual cortex.
Lee, T.S., Mumford, D.,
Journal of Optical Society of America, A. . 20(7): 1434-1448 (2003)
Week 10 (19/3)
Lectures:
Object recognition:
lecture slides (single
page)
lecture slides (4up)
Reading
Do we know what the early visual system does?, M. Carandini,
J. B. Demb, V. Mante, D. J. Tolhurst, Y. Dang, B.A. Olshausen,
J. L. Gallant, N. C. Rust,
J Neurosci 25: 10577-10597 (2005)
Further References (
details not required for NIP
course)
Hierarchical models of object recognition in cortex.
Maximilian Riesenhuber & Tomaso Poggio. Nature Neuroscience
2(11) (1999)
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., L. Wolf, S. Bilschi, M. Reisenhuber and T. Poggio.
IEEE PAMI 29(3) 411-426 (2007)
This page is maintained by
Chris Williams and
Mark van Rossum .