/   Computer Science Books /  

Information Theory Books

Advertisement

Information Theory Books

This section contains free e-books and guides on Information Theory, some of the resources in this section can be viewed online and some of them can be downloaded.

Information Theory and its applications in theory of computation

This note covers the following topics: Entropy, Kraft's inequality, Source coding theorem, conditional entropy, mutual information, KL-divergence and connections, KL-divergence and Chernoff bounds, Data processing and Fano's inequalities, Asymptotic Equipartition Property, Universal source coding: Lempel-Ziv algorithm and proof of its optimality, Source coding via typical sets and universality, joint typicality and joint AEP, discrete channels and channel capacity, Proof of Noisy channel coding theorem, Constructing capacity-achieving codes via concatenation, Polarization, Arikan's recursive construction of a polarizing invertible transformation, Polar codes construction, Bregman's theorem, Shearer's Lemma and applications, Source coding and Graph entropy, Monotone formula lower bounds via graph entropy, Optimal set Disjointness lower bound and applications, Compression of arbitrary communication protocols, Parallel repetition of 2-prover 1-round games.

Author(s):

s NAPages

Information Theory in Computer Science

This note explains the following topics: Shearer's Lemma, Entropy, Relative Entropy, Hypothesis testing, total variation distance and Pinsker's lemma, Stability in Shearer's Lemma, Communication Complexity, Set Disjointness, Direct Sum in Communication Complexity and Internal Information Complexity, Data Structure Lower Bounds via Communication Complexity, Algorithmic Lovasz Local Lemma, Parallel Repetition Theorem, Graph Entropy and Sorting.

Author(s):

s NAPages

Information Theory Lecture Notes

This is a graduate-level introduction to mathematics of information theory. This note will cover both classical and modern topics, including information entropy, lossless data compression, binary hypothesis testing, channel coding, and lossy data compression.

Author(s):

s NAPages

Lecture Notes On Information Theory

These notes provide a graduate-level introduction to the mathematics of Information Theory. Topics covered includes: Information measures: entropy, divergence and mutual information, Sufficient statistic, Extremization of mutual information, Lossless data compression, Channel coding, Linear codes, Lossy data compression, Applications to statistical decision theory, Multiple-access channel, Entropy method in combinatorics and geometry.

Author(s):

s 342Pages

Information Theory by Yao Xie

This note will explore the basic concepts of information theory. It is highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. Topics covered includes: Entropy and mutual information, Chain rules and inequalities, Data processing, Fano's inequality, Asymptotic equipartition property, Entropy rate, Source coding and Kraft inequality, Optimal code length and roof code, Huffman codes, Shannon-Fano-Elias and arithmetic codes, Maximum entropy, Channel capacity, Channel coding theorem, Differential entropy, Gaussian channel, Parallel Gaussian channel and water-filling, Quantization and rate-distortion.

Author(s):

s NAPages

Information Theory, Inference, and Learning Algorithms (David J.C. MacKay)

Currently this section contains no detailed description for the page, will update this page soon.

Author(s):

s Pages

A Discipline Independent Definition of Information (Robert M. Losee)

Currently this section contains no detailed description for the page, will update this page soon.

Author(s):

s Pages

An Introduction to Information Theory and Entropy (Tom Carter)

Currently this section contains no detailed description for the page, will update this page soon.

Author(s):

s Pages

A Short Course in Information Theory (D. MacKay)

Currently this section contains no detailed description for the page, will update this page soon.

Author(s):

s Pages

Advertisement