value iteration, in practice
Last edited: August 8, 2025Say we have a system:

- States: 4—school, internship, job, jungle
- Actions: 2—stay, graduate
create transition model
Create tables of size \(S \times S\) (that is, 4x4), one for each action. These are our transition models. Rows are the states where we took the action, columns are the states which are the results of the action, and the values are the probability of that transition happening given you took the action.
value of information
Last edited: August 8, 2025VOI is a measure of how much observing something changes your action if you are a rational agent.
The value of information a measure for how much observing an additional variable is expected to increase our utility. VOI can never be negative, and does not take into account the COST of performing the observation.
constituents
- \(o\): an observation
- \(O’\): a possible observation to run which yield \(o’_{j}\) different outcomes
requirements
\begin{equation} VOI(O’|o) = (\sum_{o’} P(o’|o) EU^{*}(o, o’)) - EU^{*}(o) \end{equation}
variance
Last edited: August 8, 2025variance (also known as second central moment) is a way of measuring spread:
\begin{align} Var(X) &= E[(X-E(X))^{2}] \\ &= E[X^{2}] - (E[X])^{2} \\ &= \qty(\sum_{x}^{} x^{2} p\qty(X=x)) - (E[X])^{2} \end{align}
“on average, how far is the probability of \(X\) from its expectation”
The expression(s) are derived below. Recall that standard deviation is a square root of the variance.
computing variance:
\begin{align} Var(X) &= E[(X - \mu)^{2}] \\ &= \sum_{x}^{} (x-\mu)^{2} p(X) \end{align}
based on the law of the Unconscious statistician. And then, we do algebra:
vc thing
Last edited: August 8, 2025- Secrets of Silicon Valley - Horowitz
- Looking for people who have feel for the problem: people need to believe in the problem
- Team: can people come with execution? people that are good at startups which are usually not good at later stage stuff
- Buy a startup and kick out the founders
- This is very typical
- Team and idea are easy to decouple
- Vetting problems
- Lack of market
- Technically insatiability
- “Unbelievable stupidity”: calcium is so cheap
- Idea goes through many morphs; getting the credit back
- People wiling to have a meeting?
- Decoupling value proposition
=> iStudio as a service
vector
Last edited: August 8, 2025A vector is an element of a vector space. They are also called a point.