Every model announcement quotes a score, and this site’s radar reports them constantly. This guide is the decoder: the four report cards you meet most often, what each one measures, and what it doesn’t tell you while it’s being waved in your face.

SWE-bench Verified: real code

What it measures. The model (rather: the model-plus-agent system) receives 500 real issues from open source Python projects, validated by hand, and must produce the fix. It passes if the project’s tests pass. No human judgement: either the code works or it doesn’t.

In thirty seconds. The percentage is the share of issues resolved. Two caveats: the score belongs to the model+scaffold pair (the same model with a different agent changes the result), and the terrain is maintenance of existing Python projects: it says little about other languages and nothing about your own project. Page: swebench.com, methodology checked on 19/07/2026.

Terminal-Bench: end-to-end work

What it measures. Eighty-nine hard terminal tasks (compiling, configuring, training, debugging) in containerised environments, each with a verification suite written beforehand. To compare models on equal footing it uses a neutral, minimal agent (Terminus 2: bash only). It is the benchmark closest to «delegating a real task to an agent».

In thirty seconds. Percentage of tasks completed autonomously. Check whether the number comes from the neutral agent or from an optimised commercial agent: they measure different things, both legitimate. Page: tbench.ai, checked the same date.

LMArena: what people like

What it measures. Real users ask the same question to two anonymous models and vote for the better answer; the votes become an Elo-style score. It measures average human preference on chat-type questions, which matters, but with documented biases: long, well-formatted answers win, answers that agree with you win, and the big labs submit many variants, raising the odds that one rises (the thesis of the «Leaderboard Illusion» paper, April 2025).

In thirty seconds. Useful for a sense of tone and pleasantness, weak as a measure of reliability on a task: a model can be likeable and get the numbers wrong. Page: lmarena.ai.

Artificial Analysis: the overview

What it measures. A composite 0-100 index aggregating nine evaluations across four equally weighted areas: agents, coding, general capability, scientific reasoning. Next to the index, the price/speed/quality curves the radar often quotes. It is the fastest way to place a new model in the landscape.

In thirty seconds. Great for the picture, but it is an average: a model mediocre in the index can excel precisely in the area you need, and vice versa. Open the per-area detail before drawing conclusions. Pages: artificialanalysis.ai and the methodology, checked the same date.

The three traps, in every announcement

Contamination. Public benchmarks end up in training data: a high score can measure memory instead of capability. Benchmarks with held-out or refreshed cases suffer less; record scores on years-old tests deserve a raised eyebrow.

Cherry-picking. The announcement quotes the benchmark where the model wins and stays silent on the rest. The short version of the checklist: if the announcement quotes a single number, ask yourself which ones are missing.

The average that hides. Two models with the same average score can have opposite profiles: one consistent, one alternating peaks and disasters. For professional use, variance matters as much as the mean, and no synthetic index shows it to you.

The final rule

Benchmarks are for getting oriented in thirty seconds, and for that they are precious. The decision, though, stays with your cases: five real examples of your work, run on the candidate models, say more than any leaderboard. A gateway’s playground turns it into a quarter-hour exercise.