Houjun Liu

Gaussian distribution

\begin{equation} \mathcal{N}(x|\mu, \Sigma) = \qty(2\pi)^{-\frac{n}{2}} |\Sigma|^{-\frac{1}{2}} \exp \qty(-\frac{1}{2} \qty(x-\mu)^{\top} \Sigma^{-1}(x-\mu)) \end{equation}

where \(\Sigma\) is positive semidefinite

conditioning Gaussian distributions

For distributions that follow Gaussian distributions, \(a, b\), we obtain:

\begin{align} \mqty[a \\ b] \sim \mathcal{N} \qty(\mqty[\mu_{a}\\ \mu_{b}], \mqty(A & C \\ C^{\top} & B)) \end{align}

meaning, each one can be marginalized as:

\begin{align} a \sim \mathcal{N}(\mu_{a}, A) \\ b \sim \mathcal{N}(\mu_{b}, B) \\ \end{align}

Conditioning works too with those terms, for \(a|b\):

\begin{align} \mu_{a|b} &= \mu_a + CB^{-1}\qty(b - \mu_{b}) \\ \sigma_{a|b} &= A - CB^{-1}C^{\top} \end{align}

standard normal density function

This is a function used to model many Gaussian distributions.

\begin{equation} \phi(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^{2}}{2}} \end{equation}

This function is the CDF of the standard normal.

standard normal density function is also symmetric:

\begin{equation} \phi(a) = 1- \phi(a) \end{equation}

Gaussian distribution


  • \(\mu\) the mean
  • \(\sigma\) the variance


\begin{equation} X \sim N(\mu, \sigma^{2}) \end{equation}

Its PDF is:

\begin{equation} \mathcal{N}(x \mid \mu, \sigma^{2}) = \frac{1}{\sigma\sqrt{2\pi}} e^{ \frac{-(x-u)^{2}}{2 \sigma^{2}}} \end{equation}

where, \(\phi\) is the standard normal density function

Its CDF:

\begin{equation} F(x) = \Phi \qty( \frac{x-\mu}{\sigma}) \end{equation}

We can’t integrate \(\Phi\) further. So we leave it as a special function.

And its expectations:

\(E(X) = \mu\)

\(Var(X) = \sigma^{2}\)

additional information

linear transformations on Gaussian

For some:

\begin{equation} Y = aX + b \end{equation}

where \(X \sim \mathcal{N}\)

We will end up with another normal \(Y \sim \mathcal{N}\) such that:

  • mean: \(au + b\)
  • variance: \(a^{2}\sigma^{2}\)

standard normal

The standard normal is:

\begin{equation} Z=\mathcal{N}(0,1) \end{equation}

mean 0, variance 1. You can transform anything into a standard normal via the following linear transform:

transformation into standard normal

\begin{equation} X \sim \mathcal{N}(\mu, \sigma^{2}) \end{equation}

and, we can shift it into a standard normal with:

\begin{equation} Z = \frac{X-\mu}{\sigma} \end{equation}

therefore, we can derive what the CDF of the normal distribution by shifting it back into the center:

\begin{equation} P(X<x) \implies P\qty(\frac{X-\mu}{\theta} < \frac{x-\mu}{\theta}) \implies P\qty(Z< \frac{x-\mu}{\theta}) = \Phi\qty(\frac{x-\mu}{\theta}) \end{equation}

normal maximizes entropy

no other random variable uses as little parameters to convey as much information

approximation of binomial distribution with normal distribution

You can use a normal distribution to approximate binomial approximation. However, be aware of a continuity correction

adding Gaussian distributions

for independent:

\begin{equation} X+Y \sim \mathcal{N}(\mu_{1}+\mu_{2}, \sigma_{1}^{2}+\sigma_{2}^{2}) \end{equation}