Information Theory, Inference, and Learning Algorithms (David J.C. MacKay)

An Introduction to Information Theory and Applications

This note explains the following topics: uncertainty and information, Efficient coding of information, Stationary processes and markov chains, Coding for noisy transmission, Complements to efficient coding of Information, Error correcting codes and cryptography.

Author(s): F Bavaud, J C Chappelier and J Kohlas

293 Pages

Lecture Notes on Information Theory by Prof. Dr. rer. nat. Rudolf Mathar

This lecture note covers introduction, Fundamentals of Information Theory, Source Coding and Information Channels.

Author(s): Prof. Dr. rer. nat. Rudolf Mathar, Institute for Theoretical Information Technology Kopernikusstr, Germany

59 Pages

Lecture Notes on statistics and information Theory

This lecture note navigates through information theory, statistics and measure theory. It covers fundamental concepts such as definitions, chain rules, data processing inequalities, and divergences and extends to optimal procedures, LeCam’s and Fano’s inequalities, and operational results like entropy and source coding. It also focus on exponential families and statistical modeling, fitting procedures, and lower bounds on testing parameters, sub-Gaussian and sub-exponential random variables, martingale methods, uniformity covering topics such as Kullback-Leibler divergence, PAC-Bayes bounds, interactive data analysis, and error bounds.

Author(s): John Duchi

464 Pages

Advanced Information Theory notes

This book contains following contents: Information Theory for Discrete Variables, Information Theory for Continuous Variables, Channel Coding, Typical Sequences and Sets, Lossy Source Coding, Distributed Source Coding, Multiaccess Channels.

Author(s): Prof Dr. sc. techn. Gerhard Kramer

180 Pages

Information Theory in Computer Science

This note explains the following topics: Shearer's Lemma, Entropy, Relative Entropy, Hypothesis testing, total variation distance and Pinsker's lemma, Stability in Shearer's Lemma, Communication Complexity, Set Disjointness, Direct Sum in Communication Complexity and Internal Information Complexity, Data Structure Lower Bounds via Communication Complexity, Algorithmic Lovasz Local Lemma, Parallel Repetition Theorem, Graph Entropy and Sorting.

Author(s): Madhu Sudan

NA Pages

A Short Course in Information Theory (D. MacKay)

Currently this section contains no detailed description for the page, will update this page soon.

Author(s):

Pages