Goal: we need to find a model that is “expressive enough”: we need to have enough parameters to help match the shape of the data we collect. to help match the shape of the data we collect.
constituents
requirements
additional information
selecting parameters
see model fitting
increasing expressiveness
mixure model
We could mix distributions into a . See Gaussian mixture model.
transforming distributions
Suppose you start with:
\begin{equation} Z \sim \mathcal{N}\qty(0,1) \end{equation}
we can sample \(k\) points \(k \sim Z\), and then transform them across a function \(x_{j}=f(k_{j})\). We now want to know the destruction of \(x_{j}\). Turns out, if \(f\) is invertible and differential, we have:
\begin{equation} p_{x}\qty(x) = p_{z}\qty(g(x)) | g’(x) | \end{equation}
where \(g(x) = f^{-1}\qty(x)\).
This new \(p_{x}\) is now the PDF of our transformed distribution.
…but how do you pick \(f\)
generative model
see generative model