Jul 7, 2026

Interesting work on predicting LLM benchmark scores, from other benchmarks - you fit a regression model over the scores on atomic, single-turn tasks to predict eval scores for agentic tasks, which are generally multi-turn and expensive to run.

Other recent work in the same vein - https://microsoft.github.io/benchpress/

The core idea is kinda expected. The matrix of LLM scores by models agains benchmark tasks should be low-rank.

• LLM performance on benchmarks is fairly general - a model that scores well on tasks A and B typically also performs well on task C.

• Some selection pressure that ensures above always holds true. Labs don't release model checkpoints until it performs similar/better at most benchmark tasks, vis-a-vis other models in the same weight class.

So, while there is some expected structure - discovering a prediction model is neat! It'd be useful to score new benchmarks with it - large prediction error on a task means it is less correlated to known benchmarks, and a signal of novelty!