Cross Entropy Method is a “conditional MLE” objective; whereby we try to maximize:

- the log prob
- of the true y labels in the training data
- given the observations

## Derivation

Recall the Bernoulli distribution, and specifically:

\begin{equation} P(Y=y) = p^{y} (1-p)^{1-y} \end{equation}

Meaning, we want to maximize:

\begin{equation} \log P(y=y) = y \log p + (1-y)\log (1-y) \end{equation}

specifically, we’d like to **minimize**:

\begin{equation} -[y \log p + (1-y)\log (1-y)] \end{equation}

## Intuition

This function should be

- smaller when the model estimate is close to correct
- bigger if the model is confused or wrong