Terminal-Bench is hosted by Stanford, the Laude Institute and the Harbor evaluation framework. Its 2.0 version was described in a January 2026 paper, “Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces”, with Mike A. Merrill, Alexander G. Shaw and Nicholas Carlini as the first of 85 authors; the paper was accepted at ICLR 2026. The current official leaderboard uses Terminal-Bench 4.1.
What it measures
Many real tasks given to AI agents happen in a terminal: compiling software, configuring servers, training a model, debugging a failing build or analysing data with command-line tools. Terminal-Bench measures whether an agent can carry out such work end to end, deciding which commands to run, reading their output and recovering from mistakes, without a human stepping in.
How it works
Each task consists of:
- a containerised environment (a sandboxed Linux system with the relevant files and software);
- an instruction describing what to achieve;
- a set of tests that check whether the job was done;
- a reference solution written by hand to prove the task is solvable.
The agent works inside the container until it decides it is finished, and the tests then decide pass or fail. The score is the percentage of tasks passed. Terminal-Bench 2.0 contained 89 tasks, each manually verified by three human reviewers; at publication, frontier models and agents scored below 65%. According to Artificial Analysis, Terminal-Bench 4.0 has 66 tasks spanning software, machine learning, science, operations, security, hardware and media, recalibrates compute and time allowances, and passes a task only if all of its tests pass.
A typical task might ask the agent to get a legacy C project compiling with a modern toolchain, or to recover data from a corrupted database file and write a summary to a specified path.
Why it matters
Terminal-Bench is one of the most direct tests of practical agentic ability: it rewards planning, tool use and persistence over long sequences of actions, the skills that matter for coding agents and automation. Because the same tasks can be run with different agent frameworks, it is also used to compare harnesses, not just models.
Limitations and criticisms
- Model plus harness. Scores reflect both the model and the agent software around it; leaderboard entries name both, and they should be compared with that in mind.
- Small task sets. With 66 to 89 tasks, each task is worth more than a percentage point, so close scores are effectively tied.
- Versions are not comparable. Versions 2.0, 2.1, 4.0 and 4.1 use different tasks or settings. Independent evaluators such as Vals AI have archived their 2.x leaderboards.
- Time and cost limits. Changing the allowed time or compute can move results substantially, which versions 4.0 and later try to standardise.
Reading the scores
The top entries above come from the official Terminal-Bench 4.1 leaderboard, which ranks model and agent combinations by resolution rate, as checked on 17 November 2026; the leaderboard data the page loads was last updated on 16 November 2026. Claude Code with Opus 5.5 (max) and Claude Code with Fable 5.2 (max) share first place at 65.8%, ahead of Antigravity CLI with Gemini 4 Carbon (high) at 64.5%. Each score carries a 95% confidence interval of about ±3 points, so the leading entries are effectively tied, and the labels in brackets are reasoning-effort settings. Artificial Analysis also runs Terminal-Bench 4.0 independently with its own fixed agent setup, so its numbers are not comparable with the official table. Since even the best systems still fail roughly a third of the tasks, the benchmark has room left; check the official site for the latest results.