Information Theory by Yao Xie
This note will explore the basic
concepts of information theory. It is highly recommended for students planning
to delve into the fields of communications, data compression, and statistical
signal processing. Topics covered includes: Entropy and mutual information,
Chain rules and inequalities, Data processing, Fano's inequality, Asymptotic
equipartition property, Entropy rate, Source coding and Kraft inequality,
Optimal code length and roof code, Huffman codes, Shannon-Fano-Elias and
arithmetic codes, Maximum entropy, Channel capacity, Channel coding theorem,
Differential entropy, Gaussian channel, Parallel Gaussian channel and
water-filling, Quantization and rate-distortion.
Author(s): Yao Xie
NA Pages