hypothesis testing is the mechanism by which a hypothesis is tested statistically.

The core logic of hypothesis testing: have a metric, do tests, calculate probability that the outcome could have happened given the metric is true.

Examples include

• t-test (for sample means)
• z-test (for sample proportions)
• chi-square test (for sample categories)

Common to all hypothesis tests are the following terms.

null hypothesis

A null hypothesis is a “no difference” hypothesis created as a part of hypothesis testing. It is usually stated as an equality.

The IBM704 is the first mass-produced floating point computation computer (“pretty much the only computer that could handle complex math”); 12,000 floating point additions per second. IBM built 123 such machines between 1955-1960.

Talks

• ICLR2025 Snell: Optimality of Scaling LLM Test-Time Compute
• ICLR2025 Mathur: MIND Adaptive Thinking with Dynamic Computation
• ICLR2025 Yue: Inference Scaling for RAG

Talks

• ICLR2025 Wu: Retrieval Head Explains Long Context

ICLR2025 Morris: contextual document embeddings

Take a bunch of sentence embeddings as input to produce a new sentence embedding that is now contextual

ICLR2025 Noukhovich: asynchronous reinforcement learning for language models

Rollout and tune concurrently

ICLR2025 Yao: CR-CTC CONSISTENCY REGULATION

CTC LOSS CAN BE MADE MORE ROBUST IF YOU REGULARIZE TO HAVE MINIMAL DIFFERENCE BETWEEN TWO AUGMENTED VIEWS OF THE SAME MEL SPECTRUM

ICLR2025 Sun: ReDeEP detecting hallucination using mechanistic interpretability

Find layers most prone to insert information, measure the information insertion using logit lens before and after passing through FFN, strong change after hallucination prone FFN means hallucination