An introduction to information Theory and Entropy
This note covers Measuring complexity, Some probability ideas, Basics of information theory, Some entropy theory, The Gibbs inequality, A simple physical example Shannons communication theory, Application to Biology, Examples using Bayes Theorem, Analog channels, A Maximum Entropy Principle, Application to Physics lasers, KullbackLeibler information measure.
Author(s): Tom Carter
139 Pages 
Information Theory Inference and Learning Algorithms
This book contains below topics, Introduction to information theory, Probability, entropy, and inference, Source coding, Noisy channel coding, Sparse graph codes, Bayesian inference and Monte Carlo methods, Neural networks, Decision theory.
Author(s): David J. C. MacKay, University of Cambridge
640 Pages
Applied Digital Information theory
This note serves as a comprehensive guide to fundamental concepts in information theory and coding. This pdf provides discrete probability theory, information theory, and coding principles. Beginning with Shannons measure of information, then delves into the efficient coding of information, the methodology of typical sequences is introduced, emphasizing the distinction between lossy and lossless source encoding. The text also discusses coding for noisy digital channels, block coding principles and tree and trellis coding principles.
Author(s): James L.Massey
153 Pages
Lecture Notes on statistics and information Theory
This lecture note navigates through information theory, statistics and measure theory. It covers fundamental concepts such as definitions, chain rules, data processing inequalities, and divergences and extends to optimal procedures, LeCams and Fanos inequalities, and operational results like entropy and source coding. It also focus on exponential families and statistical modeling, fitting procedures, and lower bounds on testing parameters, subGaussian and subexponential random variables, martingale methods, uniformity covering topics such as KullbackLeibler divergence, PACBayes bounds, interactive data analysis, and error bounds.
Author(s): John Duchi
464 Pages
An introduction to information Theory and Entropy
This note covers Measuring complexity, Some probability ideas, Basics of information theory, Some entropy theory, The Gibbs inequality, A simple physical example Shannons communication theory, Application to Biology, Examples using Bayes Theorem, Analog channels, A Maximum Entropy Principle, Application to Physics lasers, KullbackLeibler information measure.
Author(s): Tom Carter
139 Pages
Information Theory Lecture Notes
This PDF covers the following topics related to Information Theory, Introduction, Entropy, Relative Entropy, and Mutual Information, Asymptotic Equipartition Properties, Communication and Channel Capacity, Method of Types, Conditional and Joint Typicality, Lossy Compression and Rate Distortion Theory, Joint Source Channel Coding.
Author(s): Prof. Tsachy Weissman
75 Pages
Information Theory and Coding cam
The PDF covers the following topics related to Information Theory, Foundations, probability, uncertainty, information, Entropies defined, and why they are measures of information, Source coding theorem, prefix, variable, and fixedlength codes, Channel types, properties, noise, and channel capacity, Continuous information, density, noisy channel coding theorem, Fourier series, convergence, orthogonal representation, Useful Fourier theorems, transform pairs, Sampling, aliasing, Discrete Fourier transform, Fast Fourier Transform Algorithms, The quantised degreesoffreedom in a continuous signal, GaborHeisenbergWeyl uncertainty relation, Kolmogorov complexity.
Author(s): J G Daugman
75 Pages
