An Introduction to Information Theory and Applications
By F. Bavaud, J. C. Chappelier, and J. Kohlas—This long note contains a good survey of information theory and its applications. It introduces the basic ideas of uncertainty and information, then also the more practical extensions such as optimal coding schemes, followed by the theories underlying versions of stationary processes and Markov chains. Other challenges, as the note addresses, pertain to noisy transmission environments in coding. Highlighted here are several advanced topics that follow, including, importantly, error-correcting codes and cryptography. The resource will give both a theoretical background and a practical overview of how to encode, transmit, and secure information effectively. It is a very important guide for those who seek a deep understanding of information theory and how it relates to real problems of communication and data processing.
Author(s): F Bavaud, J C Chappelier and J Kohlas
293 Pages 
An Introduction to Information Theory and Applications
By F. Bavaud, J. C. Chappelier, and J. Kohlas—This long note contains a good survey of information theory and its applications. It introduces the basic ideas of uncertainty and information, then also the more practical extensions such as optimal coding schemes, followed by the theories underlying versions of stationary processes and Markov chains. Other challenges, as the note addresses, pertain to noisy transmission environments in coding. Highlighted here are several advanced topics that follow, including, importantly, error-correcting codes and cryptography. The resource will give both a theoretical background and a practical overview of how to encode, transmit, and secure information effectively. It is a very important guide for those who seek a deep understanding of information theory and how it relates to real problems of communication and data processing.
Author(s): F Bavaud, J C Chappelier and J Kohlas
293 Pages
Lecture Notes on Information Theory by Prof. Dr. rer. nat. Rudolf Mathar
The lecture notes of Prof. Dr. rer. nat. Rudolf Mathar give a clear and very compact introduction into information theory. These notes are divided into three key parts: the basics of information theory, source coding, and information channels. The introduction treats the basic notions and definitions in information theory in a very solid way. Source coding gives methods and different techniques that are used in encoding information, while the information channels section discusses the pattern in which information is carried and noise that affects it. This resource is a good pick for students and professionals who seek structure in the principles of information theory and its applications from a respected expert in the field.
Author(s): Prof. Dr. rer. nat. Rudolf Mathar, Institute for Theoretical Information Technology Kopernikusstr, Germany
59 Pages
Applied Digital Information theory
This note serves as a comprehensive guide to fundamental concepts in information theory and coding. This pdf provides discrete probability theory, information theory, and coding principles. Beginning with Shannon's measure of information, then delves into the efficient coding of information, the methodology of typical sequences is introduced, emphasizing the distinction between lossy and lossless source encoding. The text also discusses coding for noisy digital channels, block coding principles and tree and trellis coding principles.
Author(s): James L.Massey
153 Pages
Lecture Notes on statistics and information Theory
This lecture note navigates through information theory, statistics and measure theory. It covers fundamental concepts such as definitions, chain rules, data processing inequalities, and divergences and extends to optimal procedures, LeCam’s and Fano’s inequalities, and operational results like entropy and source coding. It also focus on exponential families and statistical modeling, fitting procedures, and lower bounds on testing parameters, sub-Gaussian and sub-exponential random variables, martingale methods, uniformity covering topics such as Kullback-Leibler divergence, PAC-Bayes bounds, interactive data analysis, and error bounds.
Author(s): John Duchi
464 Pages
It serves as a basis for everything, from the very basics of thermodynamics and information theory to thermodynamic potentials and distributions, principles of irreversibility, phase space evolution, and beyond. The book informs the readers about the very basics of information theory: basic notions, basic definitions, and applications. It also offers a fresh perspective on the second law of thermodynamics and quantum information, and insights into the modern view of how information theory is intertwined with the laws of physics. This book will be very useful to anyone who wants to gain an understanding of the basic issues in both thermodynamics and information theory and their intersection in current usage.
Author(s): Gregory Falkovich
165 Pages
Information Theory Lecture Notes
Prof. Tsachy Weissman's lecture notes are an excellent summary of the core topics in the subject of information theory. The document initiates with a basic overview of entropy and relative entropy, followed by mutual information and asymptotic equipartition property. Further, it discusses communications theory, channel capacity, and the method of types. It also covers key topics such as typicality-conditioned and joint, lossy compression, and rate-distortion theory. The notes also include joint source-channel coding, where there is quite a good grasp of the principles and applications of information theory. These notes will be very helpful for students and professionals looking forward to structured, comprehensive knowledge about the subject.
Author(s): Prof. Tsachy Weissman
75 Pages
