What changed
Artificial Analysis added a standalone evaluation of Cognition’s FrontierCode 1.1 on October 12. It checks more than whether a program passes tests: the evaluation also considers scope, unnecessary changes, and maintainer-defined review criteria. The original benchmark predates this report.
What the numbers mean
In the launch results, the leading configuration passed every blocking requirement in 58 percent of attempts. That is different from its average score. The evaluator uses 100 private tasks, three attempts per task, a shared agent setup, and both automated checks and an AI judge.
The workbench takeaway
Review the actual change before merging it. A fluent explanation and green tests do not establish that every edit belongs. These results come from the evaluator’s setup; we have not reproduced them, and they are not a prediction for every coding task.