Houjun Liu

SU-ENGR76 APR162024

Non-IID Sequence Can Have Smaller Entropy

For sequences that are not IID, we may have:

\begin{equation} H(X_1, \dots, X_{n)} \ll \sum_{j=1}^{n} H(X_{j}) \end{equation}

This means that for very dependent sequences:

\begin{equation} \lim_{n \to \infty} \frac{H(X_1, \dots, X_{n})}{n} \ll \sum_{j=1}^{n}H(x_{j}) \end{equation}

so to measure how good our compression is, we should use this.


a signal is, mathematically, just a function.

\begin{equation} f: \mathbb{R}^{n} \to \mathbb{R}^{m} \end{equation}

whereby the input is space (time, coordinates, etc.) and the output is the “signal” (pressure, level of gray, RGB, etc.)

here’s a sidebar:


\begin{equation} y_{f}(t) = A \sin \qty(2 \pi f t + \phi) \end{equation}

we make a whole rotation in \(\frac{1}{f}\) time, and we start at \(\phi\), and we will go to \(A\) height.

Recall sinusoids are L-periodic.

The units for sinusoids: \(t\) is seconds, \(f\) is \(\frac{1}{s}\), and amplitude is some unit.


See L-periodic and the period of the function.

triangle wave

we can construct a triangle wave by creating an Fourier Series of the shape:

\begin{equation} y(t) = \sum_{j}^{} A_{j} \sin \qty(2 \pi f_{j} t) \end{equation}


\begin{equation} A_{j} = \frac{1}{j} \end{equation}


\begin{equation} f_{j} = 2 j \end{equation}

This creates a tringle of height \(1.5\) at \(t = 0\)