# Why do FrontierCode leaderboards disagree?

Leaderboards for Cognition's FrontierCode 1.1 disagree because they run different experiments on the same 100 Main tasks. Cognition runs models through different agent harnesses, such as Anthropic's Claude Code, OpenAI's Codex, or mini-SWE-agent, and ranks each model at its best reasoning effort. Artificial Analysis runs every model in one harness, its own Stirrup, with its own LLM judge.

The two don't agree on order. As of October 18, 2026, Cognition's leaderboard puts OpenAI's GPT-6 Astra at 53.3%, above OpenAI's GPT-6.1 Sol at 50.2%. Artificial Analysis's October 12, 2026 launch results, with each model capped at 250 turns in Stirrup, reverse the pair: GPT-6.1 Sol (max) scores 49%, two points above GPT-6 Astra (max). Effort doesn't explain it, since Cognition's data file has Astra ahead at max effort too, 53.3% to 47.6%.

The harness is one difference among several. Cognition's launch post says it runs each model five times at every reasoning effort and reports the best, while Artificial Analysis averages three attempts per task, lists each effort as its own row, and grades the rubric's review criteria with OpenAI's GPT-6 Luna (medium). [Changing only the harness is enough to move the same model's score](/answers/agent-harness/why-does-the-same-model-score-differently-in-different-harnesses).

Epoch AI's benchmarking hub takes its FrontierCode results from Cognition's leaderboard, so it matches Cognition rather than adding a third measurement.

---

Sources:
- FrontierCode Leaderboard (Cognition): https://cognition.com/frontiercode (fetched 2026-10-18)
- FrontierCode leaderboard data file, per-model harness and per-effort scores (Cognition): https://cognition.com/data/frontiercode-leaderboard/data.json (fetched 2026-10-18)
- Introducing FrontierCode (Cognition, June 8, 2026): https://cognition.com/blog/frontier-code (fetched 2026-10-18)
- Announcing FrontierCode on Artificial Analysis (October 12, 2026): https://artificialanalysis.ai/articles/frontiercode (fetched 2026-10-18)
- FrontierCode 1.1 Benchmark Leaderboard (Artificial Analysis): https://artificialanalysis.ai/evaluations/frontiercode (fetched 2026-10-18)
- FrontierCode (Epoch AI Benchmarking Hub): https://epoch.ai/benchmarks/frontiercode (fetched 2026-10-18)

Source: https://tessary.ai/answers/frontiercode/why-do-frontiercode-leaderboards-disagree
More on Frontiercode: https://tessary.ai/answers/frontiercode
From Tessary, agent reliability for AI agents in production: https://tessary.ai