Key Sequence

Notation

New Concepts

• explainability
  • policy visualization
  • feature importance
  • integrated gradients
  • shapley values
• Surrogate Models
• counterfactual

Important Results / Claims

Questions

Interesting Factoids

Alignment Problem

autonomous systems will do exactly what we tell them to do… so we need to give them good instructions. This is the Alignment Problem

1. imperfect objective—underspecified objective
2. imperfect model—understanding of the world is underspecified
3. imperfect optimization—the model just didn’t solve the problem correctly

Validation Framework

High level structure:

validation_algorithm(system, spec)

system

• environment: state of the world, \(T(s’|s,a)\)
• sensor, \(O(o|s)\)
• agent, policy \(\pi\qty(a | o)\)

example: inverted pendulum

• state: \(\qty (\theta, \omega)\) of the pendulum
• observation: \(O(o|s) = \mathcal{N}\qty (o|s,\Sigma)\), Gaussian noise
• policy: consider the following proportional controller policy

\begin{equation} \pi \qty(a | o) = \begin{cases} 1, \text{if} a = -15 \tau - 8 \omega \\ 0 \end{cases} \end{equation}

Key Sequence

System Modeling

Notation

New Concepts

• model and model class
• joint probability distribution
• covariance matrix
• increasing expressiveness of your model class without inventing new distributions
  • mixure model
  • transforming distributions
  • generative model
• model fitting
  • MLE
  • MLE for Conditional Gaussian
  • MAP

Important Results / Claims

• Procedure for Building Models
• key challenges of modeling
• transforming distributions
• independent joint probability
• MLE for Gaussian

Questions

Interesting Factoids

Key Sequence

Notation

New Concepts

• probabilistic programming
• conjugate prior
• Orienteering example: aircraft collision avoidance
• evaulating model fitness
  • Quantile-Quantile plot
  • Calibration Plot
  • Turing Test
• System Specification

  • Metric

  • composite metrics
    • weighted sum method
    • goal distance

Important Results / Claims

• Marginalization Ignores Covariances

Questions

Interesting Factoids

Key Sequence

more on System Specification!

Notation

New Concepts

• composite metrics
  • weighted sum method
  • goal distance metric
• Preference Elicitation
• logical specification
  • propositional logic
  • first-order logic
  • temporal logic
    • linear temporal logic
    • Signal Temporal Logic
      • robustness (logic)

Important Results / Claims

Questions

Interesting Factoids