• hard concrete
• slimming
• zero out
• IG
• activation norm
• random

A KS test is a hypothesis test that measures if two groups of samples are drawn from the same distribution.

Kolomogorov Complexity is a “universal theory of information”. “how much information is contained in a string”

The Kolomogorov Complexity of a string \(x\) is the length of the shortest description, \(|d(x)|\)

information as description

Key idea: the more we can compress a string, the more information it contains. The amount of information in a string \(x\) is the length of the shortest description of \(x\).

aside

For some \((M,w)\), we are about to write short strings of it; how do we encode it?

Where:

\begin{equation} ||X-Y||_{\infty} = \max \{| x_{i} - y_{i} |, x \in X, y \in Y} \} \end{equation}