how many trials do you need to get r successes.

\begin{equation} P(X=n) = {{n-1} \choose {r-1}} p^{r} (1-p)^{n-r} \end{equation}

if the chance of individual success is \(p\), what’s the probability that it takes \(n\) trials to get \(r\) successes.

\begin{equation} \mathbb{E}[x] = \frac{r}{p} \end{equation}

\begin{equation} Var[x] = r \frac{{1-p}}{r^{2}} \end{equation}

see also skip-gram with negative sampling

sample some negative samples and negate them

Neoclassical Economics is a view of economics that disregards the Keynsian Politics theory of the economy needs a minder started by Milton Freedman. It believes that free market economy will prevail.

while POS Tagging assigns tags to each word, NER Tagging tags the category of usage of multi-word spans.

NER Tagging needs to label spans of text, which means that there is ambiguity in type.

BIO Tagging

BIO Tagging will tag each word: where \(B\) begins a span, \(I\), is inside a span, and \(O\) outside a span. So tags per word still apply, but we can extract span information as well.

(job - gender + gender ) = job (captial - country + country) = captial

We can think of a neural network as a roll-out of a system. For ReLU networks in particular, we can compute the exact reachable set!

Suppose we have the input set \(s_1\); let’s consider:

\begin{equation} z_1 = W_1 s_1 + b_1 \end{equation}

after one linear layer. We can then apply a nonlinear function to it. The beauty with ReLU nonlinearities is that we can split our network into one set per quadrant, and consider what ReLU will do to it.