Here is the most simple Differential Equation one could imagine:

\begin{equation} \dv{x}{t} = f(t,x) \end{equation}

Or, perhaps, we have a second order differential equation which is the same thing but in the second degree:

\begin{equation} \dv[2]{x}{t} = f\qty(t,x,\dv{x}{t}) \end{equation}

Then in which case, we have that the first most simple type of differential equation to be as follows:

\begin{equation} \dv{x}{t} = x(t) \end{equation}

If we can solve this, we can generalize this to most of other First-Order Differential Equations.

Uses a randomness temperature, which starts high for exploration and slowly decreases by one of the Simulated Annealing Schedules. Whenever you are about to take a step, each step must be “accepted” with probability given by the Metropolis Criteria.

Intuition: “sometimes, you try something that’s bad, to jump out of local optima.”

Metropolis Criteria

\begin{align} P(\text{accept}) = \begin{cases} 1, \text{if}\ \Delta y \leq 0\\ \min \qty(e^{-\frac{\Delta y}{t}}, 1), \text{otherwise} \end{cases} \end{align}

where \(t\) is the temperature; meaning—-even if the next point \(\Delta y > 0\) (meaning its bad, because it increases loss), we sometimes take that ponit still.