Information Theory

Information Theory. Entropy, channel capacity, and coding theory. The literature on information theory divides naturally along several axes: the foundational structures that organise the subject, the techniques that drive proofs and computations, the questions about classification or representation that animate current research, and the bridges to neighbouring areas of mathematics and science. The references below trace those axes through the canonical textbook treatments and recent technical contributions.

Foundations and canonical references

The standard treatments of information theory approach the subject from complementary angles. Cover, Elements of Information Theory (2006) is the anchor reference for the subject and lays out the core definitions, theorems, and worked examples that practitioners return to. MacKay, Information Theory, Inference, and Learning Algorithms (2003) offers an alternative presentation that complements the primary references and is useful for triangulating definitions and proof techniques.

Supporting and adjacent work

A number of supporting contributions sharpen specific aspects of information theory or connect it to neighbouring problems. A Mathematical Theory of Communication (Shannon, 1948) contributes to this area as one of the supporting references that inform current practice.

Open methodological questions for information theory include sharpening the bridges between foundational theory and computational practice, extending classical results to broader or more structured settings, and integrating the techniques surveyed above with adjacent mathematical disciplines. The references listed in this page are the entry points that current work builds on.