Lecture 5, Tuesday w3, 2014-09-30
=================================

**Reminders:**

* No lecture on Friday this week, 2014-11-03.
* Tutorials start next week. Please do the first tutorial question sheet.
* Ask questions on NB.


**Two things I didn't get around to mentioning last week:**

* Sketching the Binary Entropy function. Understanding its features.
* How to compute $\log\sum_i \exp(x_i)$ using standard floating point
  arithmetic, when $\exp(x_i)$ are too small or large to be represented.


**Introduction to symbol codes:**

* The idea: concatenate codewords without punctuation.
* Uniquely decodable.
* Instantaneously decodable.
* Prefix codes.
* Expected length per symbol. Compare computation to Entropy.
* Decomposition of Entropy.


Recommended reading
-------------------

We are part way through the **'week 3' slides**. I haven't quite finished
up to where it says lecture 5 ends yet. Ask on NB if anything is unclear,
or too compressed.

Ultimately you'll want to read all of **MacKay Chapter 5** in the next week
or so. For now, I've talked about the material on roughly pp90--94.