Key Sequence

Notation

New Concepts

• inference
  • Inference for Gaussian Models
• approximate inference
  • Direct Sampling
  • Likelihood Weighted Sampling
• parameter learning
  • Maximum Likelihood Parameter Learning
  • Baysian Parameter Learning
    • Dirichlet Distribution

Important Results / Claims

• “there is usually a tradeoff between the computational time you are willing to devote, and th
• Bayesian Learning on Binary Distributions

Questions

• for Likelihood Weighted Sampling, where do the conditional probability values come from

Interesting Factoids

Key Sequence

Notation

New Concepts

• Beta Distribution

Important Results / Claims

Questions

Interesting Factoids

New Concepts

• structure learning
  • K2 Algorithm
  • Local Search
• Markov Equivalence Classes
• Partially Directed Graph Search
• utility and Rational Preferences
  • lottery
  • utility elicitation
  • expected utility
  • risk aversion
• decision network
• utility functions
  • quadratic utility
  • exponential utility
  • power utility and its special case log utility
• maximum expected utility principle
• value of information

Important Results / Claims

• checking Markov Equivalence
• utility of Rational Preference
• von Neumann and Morgenstern Axioms: Axioms for checking rationality
• never have a utility function that’s infinite
• utility of a lottery
• process of observation selection

Questions

Notation

“state variables” represent the contents of the state; “state” is a complete assignment of state variables.

New Concepts

• Markov Decision Process
  • stationary Markov Decision Process
  • finite-horizon models + infinite-horizon models
• policy
  • stationary policy
  • optimal policy and optimal value function
  • policy evaluation and policy iteration
    • lookahead equation
    • Bellman Expectation Equation
    • action-value function and value-function policy
    • advantage function

Important Results / Claims

• policy evaluation methods
  • solving for the utility of a policy
  • finding the best policy
• policy iteration

Questions

• why is it d seperated

Interesting Factoids

Key Sequence

Notation

New Concepts

• Markov Decision Process
  • value iteration
• Bellman Residual
• for continuous state spaces: Approximate Value Function
  • use global approximation or local approximation methods

Important Results / Claims

• policy and utility
  • creating a good utility function / policy from instantaneous rewards: either policy evaluation or value iteration
  • creating a policy from a utility function: value-function policy (“choose the policy that takes the best valued action”)
  • calculating the utility function a policy currently uses: use policy evaluation
• kernel smoothing
• value iteration, in practice

Questions

Interesting Factoids