inference is the act of updating the distribution of a random variable based on distribution of actually observed variables:

\begin{equation} P(X|Y) \end{equation}

where \(Y\) is observed, and we want to know how likely \(X\) would therefore be.

We call the set \(X\) the “query variables”, \(Y\) as “evidence varibales”, and anything that we didn’t use which connects the two variables as “hidden variables”.

If things are not in the right order of \(X\) and \(Y\), consider the Bayes rule.

If we know that \(a,b\) are both Gaussian distributions, then we have that:

\begin{equation} \mqty[a \\ b] \sim \mathcal{N} \qty(\mqty[\mu_{a} \\mu_{b}], \mqty[A & C \\ C^{T} & B]) \end{equation}

whereby:

• \(A\) is the covariance of each element of \(A\)
• \(B\) is the covariance of each element of \(B\)
• \(C\) is the covariance of \(A\) against \(B\)

To perform inference:

\begin{equation} p(a|b) = \mathcal{N}(a | \mu_{a|B}, \Sigma_{a|b}) \end{equation}

wherby:

\begin{equation} \mu_{a|b} = \mu_{a} + CB^{-1}(b-\mu_{b}) \end{equation}

Information Retrival is trying to find material within large collections which is unstructured which satisfies an information need (of structured info).

Unstructured information has had a massive outburst after the millennium.

IMPORTANTLY: evaluating Information Retrival is based on Precision/Recall/F on information need and not the query.

For ranked system, we can come up with a curve of precision-recall curve by selecting increasing \(k\), or mean average precision.

Basic Terminology

collection

a set of documents—could by static, or dynamically added