Houjun Liu

Gaussian distribution

standard normal density function

This is a function used to model many Gaussian distributions.

\begin{equation} \phi(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^{2}}{2}} \end{equation}

This function is the CDF of the standard normal.

standard normal density function is also symmetric:

\begin{equation} \phi(a) = 1- \phi(a) \end{equation}

Gaussian distribution

constituents

  • \(\mu\) the mean
  • \(\sigma\) the variance

requirements

\begin{equation} X \sim N(\mu, \sigma^{2}) \end{equation}

Its PDF is:

\begin{equation} \mathcal{N}(x \mid \mu, \sigma^{2}) = \frac{1}{\sigma\sqrt{2\pi}} e^{ \frac{-(x-u)^{2}}{2 \sigma^{2}}} \end{equation}

where, \(\phi\) is the standard normal density function

Its CDF:

\begin{equation} F(x) = \Phi \qty( \frac{x-\mu}{\sigma}) \end{equation}

We can’t integrate \(\Phi\) further. So we leave it as a special function.

And its expectations:

\(E(X) = \mu\)

\(Var(X) = \sigma^{2}\)

additional information

linear transformations on Gaussian

For some:

\begin{equation} Y = aX + b \end{equation}

where \(X \sim \mathcal{N}\)

We will end up with another normal \(Y \sim \mathcal{N}\) such that:

  • mean: \(au + b\)
  • variance: \(a^{2}\sigma^{2}\)

standard normal

The standard normal is:

\begin{equation} Z=\mathcal{N}(0,1) \end{equation}

mean 0, variance 1. You can transform anything into a standard normal via the following linear transform:

transformation into standard normal

\begin{equation} X \sim \mathcal{N}(\mu, \sigma^{2}) \end{equation}

and, we can shift it into a standard normal with:

\begin{equation} Z = \frac{X-\mu}{\sigma} \end{equation}

therefore, we can derive what the CDF of the normal distribution by shifting it back into the center:

\begin{equation} P(X<x) \implies P\qty(\frac{X-\mu}{\theta} < \frac{x-\mu}{\theta}) \implies P\qty(Z< \frac{x-\mu}{\theta}) = \Phi\qty(\frac{x-\mu}{\theta}) \end{equation}

normal maximizes entropy

no other random variable uses as little parameters to convey as much information

approximation of binomial distribution with normal distribution

You can use a normal distribution to approximate binomial approximation. However, be aware of a continuity correction

adding Gaussian distributions

for independent:

\begin{equation} X+Y \sim \mathcal{N}(\mu_{1}+\mu_{2}, \sigma_{1}^{2}+\sigma_{2}^{2}) \end{equation}