Houjun Liu

Guilliard 2018


UAV navigation through leveraging updrafts, handling their unpredictability with POMDPs and Receeding Horizon.


  • Developed new method for low-cost POMDP online solving
  • Cool bird.

Notable Methods

two main steps

  • explore: determine thermal parameters
  • exploit: plan a trajectory to exploit the thermal


  • \(\mathcal{S}\): \(s^{u} \in \mathbb{R}^{6}\), the joint state of the UAV (2D location wrt fixed point + air speech + heading, bank, roll, altitude), and \(s^{th} \in \mathbb{R}^{2}\),the thermal status (thermal center x and y relative to UAV)
  • \(\mathcal{A}\): discretized arc trajectory segments by bank angles \(\phi_{1 \dots n}\), which executes for a fixed \(T_{A}\) seconds
  • \(\mathcal{T}\): Gaussian of \(s^{u}\) over the dynamics of the UAV, and over fixed noise covariance \(Q\)
  • \(\mathcal{R}\): \(h_{s’}-h_{s}\), the change in altitude….
  • \(\mathcal{O}\): senor readings
  • \(O(a, s’, o)\): fixed noise covariance \(R\)
  • \(b_0\): product of two Gaussian of the UAV’s position and the belief about the underlying thermals
  • \(update(b,a,o)\): EKF

modeling assumptions:

  • thermal consistency: the world model change frequency less than control
  • thermal stationarity: thermal doesn’t move against surrounding air
  • no pitch angle control: reward hacking may happen
  • no turbulence: thermal doesn’t sang horizontal displacements


The exact solution to the POMDP as proposed makes aggressive decisions in order to simplify costs to run on a PixHawk.

We need to explicitly build in a exploration/exploitation tradeoff.

Key Figs

comparison against ardusoar: EKF + just circling

ArduPilot’s implementation is worse

New Concepts

POMDSoar, the soring mechanism