ACL2025 Zhang: FaithfulRAG: Fact level conflict modeling

Key insight: RAG performance degrades wen model has context and parametric knowledge mismatch, identifying those and use three step iterative method to improve context faithfulness.

ACL2025 Ding: LLM reasoning capability via scalable question synthesis

Key insight: generate free-from questions conditioned only in BOS, then distill and DPO to get a nice question generation dataset and directly fine tune

ACL2025 Wen: synthetic data strategy on domain specific retrieval

Key insight: train your models enough to memorize the context of a specific domain and therefore be able to recall better in particular using document based IDs

• ACL2025 Li: TokAlign Token Alignment
• ACL2025 Pagoni: Patches Scale Better Than Tokens

• ACL2025 Huang: Making in Multi-Hop QA

One-Liner

“Patches in groups of tokenization scale better than tokens”

Motivation / Novelty

• typical byte-level LMs don’t are very expensive because many tokens
• its hard to go beyond 4-6 bytes per token: Zipf’s Law
• so, we model them as token patches

Notable Methods

token patch

“how do we segment the byte sequence into patches?” — insight: group predicable tokens after every hard choice! i.e., once you train a model, there are “obvious”