ICML2026 Zhang: generalist value model

Use embedding similarity model with respect to previous group performance as the value model, such the value model can take previous performance as well as rollout as input, such hard to predict values more generally

ICML2026 Jayalath: compute as teacher

Consensus voting among multiple rollout used as supervisory target for reward and gold derivation

ICML2026 Mirvakabova: dirchlet prior sampling

For expert upcycling, initialize router based on weighted dirchlet priors instead of uniform

ICML2026 Nadgir: how does chain of thought decompose complex tasks

There is a optimal depth of tree for complex tasks in CoT

ICML2026 Kong: understanding compositional generalizations

SFT memorizes, which introduces modules, RL breaks it down into a reusable components, prove this by ablation into each component and measure task gain under synthetic setting

ICML2026 Attinas: measuring the impact of tokenizer choice

Suddenly finally did the thing where they measure a different tokenizers under the same training recipe

ICML2026 Zuhri: predicting the order of multi token prediction

Multi token prediction predict order as well as the actual tokens improves performance

ICML2026 Tastan: mixture of slimmable experts

In addition to a mixture of experts, also makes some smaller experts i.e. decide how much of each expert to actually activate

ICML2026 Wang: conformal thinking

UCB based calibration for maximum thinking to reduce overthinking for fixed budget conditions

ICML2026 Fu: attention forges native mixture of experts in attention layers sink aware training

Vanilla attention nominates some heads to be always important to activated, which is bad utilization, similarly, sink parameters in attention a la GPT still result in lower balancing, add auxiliary head usage loss like moe load balance

ICML2026 Sun: anatomy of massive activation

Large activation after RMS norm has magnitude normalized way but directions still large, essentially acting as a basis vector which is active only in one channel, inducing attention sinks. These sinks can act as a conditional dating mechanism to kill a bunch of heads that you don’t want to select (in the zero column) and also to kill thinking and ambiguity in short context situations

ICML2026 Shen: efficient reasoning with hidden thinking

Progressively distill tokens down by taking the original pre-train model and substituting cot traces with less and less tokens. Also train a decoder that would decode out these tokens auto aggressively into full CoT.

ICML2026 Isik: names don’t matter

LMs are bad extracting hierarchical structure, so, before masking of non-structural elements one for each structural part, and average mask tokens together. Therefore you enrich embeddings and it’s now more robust to name changes

ICML2026 Rajesh: panini

Higherarical memory structure by encoding entity structure corefs into graph, merging graph nodes, and producing coherent ontology

ICML2026 Putzky: Float8@2Bits

Use entropy coding to encode makes better quality

ICML2026 Ma: mu-p

Small to large hyper parameter transfer

ICML2026 Zeng: ponder LM 2

Stick the output embeddings into the input, at train time pay three times the cost to basically do this iteratively, yay

ICML2026 Fein: one bias after another

Extract last layer activation and find and project out undesirable things wrt a linear separable plane