_index.org

Logit Probe

Last edited: August 8, 2025

Goals

Motivation: it is very difficult to have an interpretable, causal trace of facts. Let’s fix that.

Facts

It is also further difficult to pull about what is a “fact” and what is a “syntactical relation”. For instance, the task of

The Apple iPhone is made by American company <mask>.

is different and arguably more of a syntactical relationship rather than factually eliciting prompt than

The iPhone is made by American company <mask>.

For our purposes, however, we obviate this problem by saying that both of these cases are a recall of the fact triplet <iPhone, made_by, Apple>. Even despite the syntactical relationship established by the first case, we define success as any intervention that edits this fact triplet without influencing other stuff of the form:

logspace

Last edited: August 8, 2025

\begin{equation} L = \text{SPACE}\qty(\log n) \end{equation}

For time, the gold standard for languages with \(\geq n\) to read input is \(\text{TIME}\qty(n)\) or at best \(\text{TIME}\qty(n^{k})\).

For space, the gold standard for languages with \(\geq n\) characters is \(\text{SPACE}\qty(\log n)\), because to have pointers, store things, etc., will take this much.

additional information

example

Here are some logspace algorithms.

0 and 1

\begin{equation} A = \qty {0^{m}1^{m}: m \in \mathbb{N}} \end{equation}

palendromes

We can solve it by keeping track of length of input, and then check \(x [i] = x[n-i+1]\)

loop invariant

Last edited: August 8, 2025

lossless compression

Last edited: August 8, 2025

lossy compression

Last edited: August 8, 2025