University Homepage School Homepage School Contacts School Search

Lecture 5, Tuesday w3, 2014-09-30

Reminders:

• No lecture on Friday this week, 2014-11-03.
• Tutorials start next week. Please do the first tutorial question sheet.
• Ask questions on NB.

Two things I didn’t get around to mentioning last week:

• Sketching the Binary Entropy function. Understanding its features.
• How to compute log∑ _iexp(x_i) using standard floating point arithmetic, when exp(x_i) are too small or large to be represented.

Introduction to symbol codes:

• The idea: concatenate codewords without punctuation.
• Uniquely decodable.
• Instantaneously decodable.
• Prefix codes.
• Expected length per symbol. Compare computation to Entropy.
• Decomposition of Entropy.

Recommended reading

We are part way through the ‘week 3’ slides. I haven’t quite finished up to where it says lecture 5 ends yet. Ask on NB if anything is unclear, or too compressed.

Ultimately you’ll want to read all of MacKay Chapter 5 in the next week or so. For now, I’ve talked about the material on roughly pp90–94.

---

This page maintained by Iain Murray.
Last updated: 2014/09/30 14:23:09