Key Sequence

Notation

New Concepts

• global approximation
• online planning
  • Rollout with Lookahead
  • Forward Search
  • Branch and Bound
  • monte-carlo tree search
• open loop planning

Important Results / Claims

• Rollout Policy
• monte-carlo exploration
• open-loop planning vs close-loop planning

Questions

Interesting Factoids

Big day. Policy Gradient.

New Concepts

• Approximate Policy Evaluation and Roll-out utility
• Policy Optimization methods:
  • Local Policy Search (aka Hooke-Jeeves Policy Search)
  • Genetic Policy Search
  • Cross Entropy Method
  • Policy Gradient, Regression Gradient and Likelyhood Ratio Gradient
  • Reward-to-Go

Important Results / Claims

• monte-carlo policy evaluation
• Finite-Difference Gradient Estimation
• Linear Regression Gradient Estimate

Questions for next office hour

Key Sequence

Notation

New Concepts

• Restricted Gradient and Actor-Critic
• Exploration and Exploitation with Binary Bandit
  • Undirected Exploration (Explore-then-commit)
  • Directed Exploration (Softmax Method, Quantile Exploration, UCB 1, Posterior Sampling)

Important Results / Claims

• Bayesian Model Estimation and greedy action
• epsilon-greedy exploration with decay

Questions

Interesting Factoids

See Double Envelope Problem

Key Sequence

• we introduced Decision Making, models of decision making and gave some examples
• we introduced different types of uncertainty
• in the reading, we introduced the Course Outline

New Definitions

• Decision Making
• uncertainty

Questions for Jana

• how are planning methods different from explicit programming methods?
• “Sequential Decision Process”
• Partially observable Markov decision process (POMDP)