ACL2025 Katz: segment based attention masking
Key insight: allow by directional attention
ACL2025 Monodorf: exploring modular sturctures transformer based language models
Key insight: learn circuit compositions by learning a binary mask for both faithfulness and scarcity
ACL2025 Li: some more samples of next token prediction
Key insight: when there’s a high difference between generation probability and ground truth, those samples when intervene will cause a more dramatic effect
ACL2025 Kim: counterfactual consistency prompting
Key insight: prompt with counter factual for temporal order to be able to be more consistent temporally
ACL2025 Peng: guiding retrieval augmented reasoning through process reward tree search
Key insight: breakdown original query into sub parts and answers, using Monte Carlo tree search, with gold traces, then fine tune reasoning model
ACL2025 Liu: Judge as a Judge: Improving the Evaluation of Retrieval-Augmented Generation
Key insight: is a panel of different LLMs to be synthesized together to improve retrieve augmented generation reward signal
Good baseline; also GLM-4
ACL2025 Huang: low entropy watermark detection
Key insight: current red green detection underestimate low entry positions because it can’t be altered as much, fixed with Bayes rule