Computer Science BooksInformation Theory Books

Information Theory by Yao Xie

Advertisement

Information Theory by Yao Xie

Information Theory by Yao Xie

This note will explore the basic concepts of information theory. It is highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. Topics covered includes: Entropy and mutual information, Chain rules and inequalities, Data processing, Fano's inequality, Asymptotic equipartition property, Entropy rate, Source coding and Kraft inequality, Optimal code length and roof code, Huffman codes, Shannon-Fano-Elias and arithmetic codes, Maximum entropy, Channel capacity, Channel coding theorem, Differential entropy, Gaussian channel, Parallel Gaussian channel and water-filling, Quantization and rate-distortion.

Author(s):

sNA Pages
Similar Books
Lecture Notes on Information Theory by Prof. Dr. rer. nat. Rudolf Mathar

Lecture Notes on Information Theory by Prof. Dr. rer. nat. Rudolf Mathar

This lecture note covers introduction, Fundamentals of Information Theory, Source Coding and Information Channels.

s59 Pages
Lecture Notes on statistics and information Theory

Lecture Notes on statistics and information Theory

This lecture note navigates through information theory, statistics and measure theory. It covers fundamental concepts such as definitions, chain rules, data processing inequalities, and divergences and extends to optimal procedures, LeCam’s and Fano’s inequalities, and operational results like entropy and source coding. It also focus on exponential families and statistical modeling, fitting procedures, and lower bounds on testing parameters, sub-Gaussian and sub-exponential random variables, martingale methods, uniformity covering topics such as Kullback-Leibler divergence, PAC-Bayes bounds, interactive data analysis, and error bounds.

s464 Pages
Advanced Information Theory notes

Advanced Information Theory notes

This book contains following contents: Information Theory for Discrete Variables, Information Theory for Continuous Variables, Channel Coding, Typical Sequences and Sets, Lossy Source Coding, Distributed Source Coding, Multiaccess Channels.

s180 Pages
Information Theory and its applications in theory of computation

Information Theory and its applications in theory of computation

This note covers the following topics: Entropy, Kraft's inequality, Source coding theorem, conditional entropy, mutual information, KL-divergence and connections, KL-divergence and Chernoff bounds, Data processing and Fano's inequalities, Asymptotic Equipartition Property, Universal source coding: Lempel-Ziv algorithm and proof of its optimality, Source coding via typical sets and universality, joint typicality and joint AEP, discrete channels and channel capacity, Proof of Noisy channel coding theorem, Constructing capacity-achieving codes via concatenation, Polarization, Arikan's recursive construction of a polarizing invertible transformation, Polar codes construction, Bregman's theorem, Shearer's Lemma and applications, Source coding and Graph entropy, Monotone formula lower bounds via graph entropy, Optimal set Disjointness lower bound and applications, Compression of arbitrary communication protocols, Parallel repetition of 2-prover 1-round games.

sNA Pages
Information Theory Lecture Notes

Information Theory Lecture Notes

This is a graduate-level introduction to mathematics of information theory. This note will cover both classical and modern topics, including information entropy, lossless data compression, binary hypothesis testing, channel coding, and lossy data compression.

sNA Pages
Information Theory by Yao Xie

Information Theory by Yao Xie

This note will explore the basic concepts of information theory. It is highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. Topics covered includes: Entropy and mutual information, Chain rules and inequalities, Data processing, Fano's inequality, Asymptotic equipartition property, Entropy rate, Source coding and Kraft inequality, Optimal code length and roof code, Huffman codes, Shannon-Fano-Elias and arithmetic codes, Maximum entropy, Channel capacity, Channel coding theorem, Differential entropy, Gaussian channel, Parallel Gaussian channel and water-filling, Quantization and rate-distortion.

sNA Pages

Advertisement