Non-IID Sequence Can Have Smaller Entropy
For sequences that are not IID, we may have:
\begin{equation} H(X_1, \dots, X_{n)} \ll \sum_{j=1}^{n} H(X_{j}) \end{equation}
This means that for very dependent sequences:
\begin{equation} \lim_{n \to \infty} \frac{H(X_1, \dots, X_{n})}{n} \ll \sum_{j=1}^{n}H(x_{j}) \end{equation}
so to measure how good our compression is, we should use this.
signal
a signal is, mathematically, just a function.
\begin{equation} f: \mathbb{R}^{n} \to \mathbb{R}^{m} \end{equation}
whereby the input is space (time, coordinates, etc.) and the output is the “signal” (pressure, level of gray, RGB, etc.)
here’s a sidebar:
sinusoid
\begin{equation} y_{f}(t) = A \sin \qty(2 \pi f t + \phi) \end{equation}
we make a whole rotation in \(\frac{1}{f}\) time, and we start at \(\phi\), and we will go to \(A\) height.
Recall sinusoids are L-periodic.
The units for sinusoids: \(t\) is seconds, \(f\) is \(\frac{1}{s}\), and amplitude is some unit.
L-periodic
See L-periodic and the period of the function.
triangle wave
we can construct a triangle wave by creating an Fourier Series of the shape:
\begin{equation} y(t) = \sum_{j}^{} A_{j} \sin \qty(2 \pi f_{j} t) \end{equation}
where:
\begin{equation} A_{j} = \frac{1}{j} \end{equation}
and:
\begin{equation} f_{j} = 2 j \end{equation}
This creates a tringle of height \(1.5\) at \(t = 0\)