\(\mathbb{F}^n\) is the set of all lists of length \(n\) with elements of \(\mathbb{F}\). These are a special case of matricies.

Formally—

\begin{equation} \mathbb{F}^n = \{(x1,\ldots,x_n):x_j\in\mathbb{F}, \forall j =1,\ldots,n\} \end{equation}

For some \((x_1,\ldots,x_n) \in \mathbb{F}^n\) and \(j \in \{1,\ldots,n\}\), we say \(x_j\) is the \(j^{th}\) coordinate in \((x_1,\ldots,x_n)\).

additional information

addition in \(\mathbb{F}^n\)

Addition is defined by adding corresponding coordinates:

\begin{equation} (x1,\ldots,x_n) + (y_1,\ldots,y_n) = (x_1+y_1, \ldots,x_n+y_n) \end{equation}

addition in \(\mathbb{F}^n\) is commutative

If we have \(x,y\in \mathbb{F}^n\), then \(x+y = y+x\).

We define a set \(\mathbb{F}^{s}\), which is the set of unit functions that maps from any set \(S\) to \(\mathbb{F}\).

closeness of addition

\begin{equation} (f+g)(x) = f(x)+g(x), \forall f,g \in \mathbb{F}^{S}, x \in S \end{equation}

closeness of scalar multiplication

\begin{equation} (\lambda f)(x)=\lambda f(x), \forall \lambda \in \mathbb{F}, f \in \mathbb{F}^{S}, x \in S \end{equation}

commutativity

inherits \(\mathbb{F}\) (for the codomain of functions \(f\) and \(g\))

associativity

inherits \(\mathbb{F}\) for codomain or is just \(\mathbb{F}\) for scalar

in probability, a factor \(\phi\) is a value you can assign to each distinct value in a discrete distribution which acts as the probability of that value occurring. They are considered parameters of the discrete distribution.

If you don’t have discrete variables, factors allow you to state \(p(x|y)\) in terms of a function \(\phi(x,y)\).

See also Rejection Sampling

factor operations

factor product

\begin{equation} \phi_{3} (x,y,z) = \phi_{1} (x,y) \cdot \phi_{2}(y,z) \end{equation}

factor marginalization

\begin{equation} \phi(x) = \sum_{y=Y} \phi(x,y) \end{equation}

Motivation

Multiple agents need to collaborate to achieve common goal.

Joint Utility Maximization: maximize the joint utility between various agents.

Possible Approaches

• Using a traditional MDP: an MDP considers “action” as a joint action between all agents (exponential blow up because the agent actions multiply)
• Local Optimization: share rewards/values among agents
• Local Optimization: search and maximize joint utility explicitly (no need to model the entire action space)

Problems with single Reward Sharing:

For a trajectory \(p\qty(\tau)\), the failure distribution is $p \qty(τ | τ ¬ ∈ψ)$—the probability of a particular trajectory given that its a failure:

\begin{equation} p \qty( \tau \mid \tau \not \in \psi) = \frac{\mathbb{1}\qty {\tau \not \in \psi} p\qty(\tau)}{ \int \mathbb{1}\qty {\tau \not \in \psi} p\qty(\tau) \dd{\tau}} \end{equation}

This bottom integral could be very difficult to compute; but the numerator may take a bit more work to compute!

So ultimately we can also give up and don’t normalize (and then use systems that allows us to draw samples from unnormalized probability densities: