information
- information source can be modeled as a random variable
- surprise
- information value and entropy
Source Coding
- non-singular code
- uniquely decodeable
- Prefix-Free code, which is uniquely decodeable and instantaneously decodable
- Average Codeword Length
- generating Prefix-Free code with Huffman Coding
- Block Coding
- Boundedness of Block Coding and why it saves space!
- Diadic Source
- Shannon’s Source-Coding Theorem
- Non-IID Sequence Can Have Smaller Entropy (which is why we use Entropy Rate of the Source as the measure of how good a code is)