Information Theory, Inference, and Learning Algorithms (David J.C. MacKay)

Advanced Information Theory notes

This book contains following contents: Information Theory for Discrete Variables, Information Theory for Continuous Variables, Channel Coding, Typical Sequences and Sets, Lossy Source Coding, Distributed Source Coding, Multiaccess Channels.

Author(s): Prof Dr. sc. techn. Gerhard Kramer

180 Pages

Information Theory by Y. Polyanskiy

This PDF covers the following topics related to Information Theory : Information measures, Lossless data compression, Binary hypothesis testing, Channel coding, Lossy data compression, Advanced topics.

Author(s): Y. Polyanskiy, Y. Wu

295 Pages

Information Theory in Computer Science

This note explains the following topics: Shearer's Lemma, Entropy, Relative Entropy, Hypothesis testing, total variation distance and Pinsker's lemma, Stability in Shearer's Lemma, Communication Complexity, Set Disjointness, Direct Sum in Communication Complexity and Internal Information Complexity, Data Structure Lower Bounds via Communication Complexity, Algorithmic Lovasz Local Lemma, Parallel Repetition Theorem, Graph Entropy and Sorting.

Author(s): Madhu Sudan

NA Pages

Information Theory Lecture Notes

This is a graduate-level introduction to mathematics of information theory. This note will cover both classical and modern topics, including information entropy, lossless data compression, binary hypothesis testing, channel coding, and lossy data compression.

Author(s): Prof. Yury Polyanski

NA Pages

Information Theory by Yao Xie

This note will explore the basic concepts of information theory. It is highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. Topics covered includes: Entropy and mutual information, Chain rules and inequalities, Data processing, Fano's inequality, Asymptotic equipartition property, Entropy rate, Source coding and Kraft inequality, Optimal code length and roof code, Huffman codes, Shannon-Fano-Elias and arithmetic codes, Maximum entropy, Channel capacity, Channel coding theorem, Differential entropy, Gaussian channel, Parallel Gaussian channel and water-filling, Quantization and rate-distortion.

Author(s): Yao Xie

NA Pages

Information Theory, Inference, and Learning Algorithms (David J.C. MacKay)

Currently this section contains no detailed description for the page, will update this page soon.

Author(s):

Pages