We can think of a neural network as a roll-out of a system. For ReLU networks in particular, we can compute the exact reachable set!

Suppose we have the input set \(s_1\); let’s consider:

\begin{equation} z_1 = W_1 s_1 + b_1 \end{equation}

after one linear layer. We can then apply a nonlinear function to it. The beauty with ReLU nonlinearities is that we can split our network into one set per quadrant, and consider what ReLU will do to it.

Neural Network Unit

A real-valued vector as input, each multiplied by some weights, summed, and squashed by some non-linear transform.

\begin{equation} z = w\cdot x + b \end{equation}

and then, we will squash this using it as an “activation”

\begin{equation} y = \sigmoid(z) \end{equation}

One common activation is sigmoid. So, one common formulation would be:

\begin{equation} y = \frac{1}{1+\exp (- (w \cdot x + b))} \end{equation}

Tanh

\begin{equation} y(z) = \frac{e^{z} - e^{-z}}{e^{z}+e^{-z}} \end{equation}

scene representation

artificial vs biological intelligence

• Humans are few-shot learners (“sample efficiency”)

• Humans can easily fine-tunable (“transfer flexibility”)

• Human knowledge can transfer easily

• AI are many-shot learners (“sample inefficiency”)

• AI are specialized

• AI is more precise, and can hold a lot in cache

biological learning

biological learning is mostly unsupervised, and yte can generalize

visual processing

a neutral stability ( mar ) condition in Differential Equations means that a function is neither stable nor unstable: it does not

See: https://en.wikipedia.org/wiki/Marginal_stability

Election between Hayes vs Tildon was very close. Democrats gave Republicans Hayes, but then asked the Republican millitary to leave the South and hence they have no way of enforcing the rights.

Redeemer Governments

Democrats put in systems to relegate African Americans to second-class citizenship into the south. Lynchings became the weapon of choice of enforcing Jim Crow.

• Within 20 years, Jim Crow became implemented by every state
• 1896 Plessy vs Ferguson upholding the process of segregation
• Convict leasing: convicts’ labour was leased to create infrastructure

Economic transformation: put in sharecropping (crops in lieu or in addition to rent) and convict leasing. This is essentially modern slavery because debt is used as a process to enslave people as they will never actually be paid enough to pay back debt.