Can I run FrontierCode on my own coding agent?

No: Cognition keeps FrontierCode’s tasks private to stop models from training on them, and there’s no public dataset or harness you can point your own agent at. Model creators can request an evaluation from Cognition itself, which runs the attempt and reports back the score.

Artificial Analysis runs a second, independent version instead of building your own: it evaluates every model in its own open-source Stirrup harness, using the 100-task Main set, three attempts per task, and GPT-6 Luna as the judge. That’s the number you can actually compare a model against across labs, since Cognition’s own leaderboard tests that harness, not yours. That gap between a vendor’s harness and an independent one is the same reason the same model scores differently depending on who’s running it.

Neither leaderboard tells you how your own coding agent, with your own repo, tools, and prompts, would do on a task like FrontierCode’s. The tasks are drawn from real open-source maintainers’ own review criteria, which is a reasonable proxy, but a proxy is all a private benchmark can give you.