Lovely, well known result:

\begin{equation} \dv x \int_{a}^{x} f(t)\dd{t} = f(x) \end{equation}

for any fixed \(a\). This is because that’s functionally using \(a\) as a \(+C\) term.

The dimension of the null space plus the dimension of the range of a Linear Map equals the dimension of its domain.

This also implies that both the null space (but this one’s trivial b/c the null space is a subspace of the already finite-dimensional domain) and the range as well is finite-dimensional.

constituents

• \(T \in \mathcal{L}( V,W )\)
  • finite-dimensional \(V\) (otherwise commenting on computing its dimension doesn’t make sense)

requirements

\begin{equation} \dim V = \dim null\ T + \dim range\ T \end{equation}

fusion in machine learning is the process of adding features or encoding.

late fusion

late fusion adds features together to a model in a multi-modal approach by first embedding the features separately

early fusion

early fusion adds features together to a model in a multi-modal approach by concatenating the features first then embedding

Main problem: joint actions and observations are exponential by the number of agents.

Solution: Smaple-based online planning for multiagent systems. We do this with the factored-value POMCP.

• factored statistics: reduces the number of joint actions (through action selection statistics)
• factored trees: reduces the number of histories

Multiagent Definition

• \(I\) set of agents
• \(S\) set of states
• \(A_{i}\) set of states for each agent \(i\)
• \(T\) state transitions
• \(R\) reward function
• \(Z_{i}\) joint observations for each agents
• \(O\) set of observations

Coordination Graphs

you can use sum-product elimination to shorten the Baysian Network of the agent Coordination Graphs (which is how agents influnece each other).

Motivation

Its the same. It hasn’t changed: curses of dimensionality and history.

Goal: to solve decentralized multi-agent MDPs.

Key Insights

1. macro-actions (MAs) to reduce computational complexity (like hierarchical planning)
2. uses cross entropy to make infinite horizon problem tractable

Prior Approaches

• masked Monte Carlo search: heuristic based, no optimality garantees
• MCTS: poor performance

Direct Cross Entropy

see also Cross Entropy Method

1. sample a value function \(k\)
2. takes \(n\) highest sampled values
3. update parameter \(\theta\)
4. resample until distribution convergence
5. take the best sample \(x\)

G-DICE

1. create a graph with exogenous \(N\) nodes, and \(O\) outgoing edges (designed before)
2. use Direct Cross Entropy to solve for the best policy

Results

1. demonstrates improved performance over MMCS and MCTS
2. does not need robot communication
3. garantees convergence for both finite and infiinte horizon
4. can choose exogenous number of nodes in order to gain computational savings