Ingredients:

• \(\mathcal{P}\) problem (states, transitions, etc.)
• \(\pi\) a Rollout Policy
• \(d\) depth (how many next states to look into)—more is more accurate but slower

Use the greedy policy at each state by using the Rollout procedure to estimate your value function at any given state.

Rollout

Rollout works by hallucinating a trajectory and calculating the reward.

For some state, Rollout Policy, and depth…

1. let ret be 0; for i in range depth
   1. take action following the Rollout Policy
   2. obtain a sample of possible next state (weighted by the action you took, meaning an instantiation of \(s’ \sim T(\cdot | s,a)\)) and reward \(R(s,a)\) from current state
   3. ret += gamma^i * r
2. return ret

Rollout Policy

A Rollout Policy is a default policy used for lookahead. Usually this policy should be designed with domain knowledge; if not, we just use a uniform random next steps.

Ronald Reagan is a president of the United States. He rises a wave of the New Right.

• Comes out of Hollywood and was CA governor
• Reagan was a democrat, but McCarthyism lead him
• Reagan was an FBI informer for McCarthyism investigations
• Reagan was the first two-term president since 1961, was able to maintain more power compared to others
• “The Great Communicator”

Reagan politics

“Government isn’t the solution to the problem, its the problem.”

The Rosa Parks bus incident is the instigator which needed to act on an issue to challenge the civil rights movement.

She participated in many civil rights agitations, and became the instigator .

Rosetta is a set of physical-based protein folding models.

protein binding with Rosetta

• check a protein surface
• check how protein side-chains interact with the binding surface

peptide binding with Rosetta

The difficulty with this is that we don’t know what the overall tertiary structure of a group of peptides are; unlike whole protein binding.

sequence-specific DNA binding

???

more!

You take something like a trimer; you shove a peptide between each “point”, and boom structal change to a quadromer