ACL2025 Katz: segment based attention masking

Key insight: allow by directional attention

ACL2025 Monodorf: exploring modular sturctures transformer based language models

Key insight: learn circuit compositions by learning a binary mask for both faithfulness and scarcity

ACL2025 Li: some more samples of next token prediction

Key insight: when there’s a high difference between generation probability and ground truth, those samples when intervene will cause a more dramatic effect

ACL2025 Kim: counterfactual consistency prompting

Key insight: prompt with counter factual for temporal order to be able to be more consistent temporally

Capecitabmine => 5-Fluoropyrimidine => Cancer cell death.

Quality of taking a particular value at a function—“expected discounted return when following a policy from \(S\) and taking \(a\)”:

\begin{equation} Q(s,a) = R(s,a) + \gamma \sum_{s’} T(s’|s,a) U(s’) \end{equation}

where, \(T\) is the transition probability from \(s\) to \(s’\) given action \(a\).

value function

Therefore, the utility of being in a state (called the value function) is:

\begin{equation} U(s) = \max_{a} Q(s,a) \end{equation}

“the utility that gains the best action-value”

value-function policy

A value-function policy is a policy that maximizes the action-value