Information theory lecture log
This lecture log is an experiment. I'm wondering if it's useful to list what
I think the key points of a lecture were. And what questions you should ask
yourself to check understanding. For interest, I may also point to other
resources beyond the course. I'll try to point to the notes and the textbook,
rather than repeat them.
The PDF versions are on NB for comments and questions. The NB links need you
to have signed up to the class forum, and be logged in
to it. Or ask questions on any of the other class materials by going via the
NB main page (logging in if necessary).
Lectures:
- Lecture 1, Tuesday w1, 2014-09-17,
text,
html,
pdf,
NB.
Introduction.
- Lecture 2, Friday w1, 2014-09-19,
text,
html,
pdf,
NB.
Block coding idea and Binomials.
- Lecture 3, Tuesday w2, 2014-09-23,
text,
html,
pdf,
NB.
CLT and Introduction to Information.
- Lecture 4, Friday w2, 2014-09-26,
text,
html,
pdf,
NB.
Source Coding Theorem.
- New note: An overview of the source coding theorem,
pdf,
2-up pdf,
NB.
- Lecture 5, Tuesday w3, 2014-09-29,
text,
html,
pdf,
NB.
Introduction to symbol codes.
- No lecture on Friday w3, 2014-10-03.
- Lecture 6, Tuesday w4, 2014-10-07,
text,
html,
pdf,
NB.
More symbol coding.
- Lecture 7, Friday w4, 2014-10-10,
text,
html,
pdf,
NB.
Convexity and applying Jensen's inequality.
- Lecture 8, Tuesday w5, 2014-10-14,
text,
html,
pdf,
NB.
Stream codes: arithmetic coding
- Lecture 9, Friday w5, 2014-10-17,
text,
html,
pdf,
NB.
More arithmetic coding. Inference and prediction.
- Monday 2014-10-20: assignment released.
- Lecture 10, Tuesday w6, 2014-10-21,
text,
html,
pdf,
NB.
More on prediction.
- Lecture 11, Friday w6, 2014-10-21,
text,
html,
pdf,
NB.
PPM. Started part 2 of the course, on communication.
- Lecture 12, Tuesday w7, 2014-10-28,
text,
html,
pdf,
NB.
Entropies and Mutual Information.
- Lecture 13, Friday w7, 2014-10-31,
text,
html,
pdf,
NB.
I(X;Y), optimal and nonconfusable inputs, checksums.
- Lecture 14, Tuesday w8, 2014-11-04,
text,
html,
pdf,
NB.
Block codes.
- Lecture 15, Friday w8, 2014-11-07,
text,
html,
pdf,
NB.
Block and fountain codes for the erasure channel.
- Lecture 16, Tuesday w9, 2014-11-11,
text,
html,
pdf,
NB.
The noisy channel coding theorem (proof).
- Lecture 17, Friday w9, 2014-11-14,
text,
html,
pdf,
NB.
Code distance. Sketch of LDPC with sum-product. (Mention of Reed Solomon.)
- That's the end of the examinable part of the course.
This page maintained by Iain Murray.
Last updated: 2014/11/15 09:01:06
