# Can I start running agent evals before I have a labeled dataset?

Yes. You don't need a hand-labeled dataset to start grading an agent: write a rubric for what a good and a bad answer look like, and an LLM judge can score real production traces against it today. Nothing has to exist before you can start.

The labeled set builds itself as a side effect of doing that. Traces the judge flags, or that a human overrides, go into review, and once someone confirms what the agent should have done, that trace becomes a labeled case grounded in a real failure rather than one invented at a desk.

What this doesn't give you is confidence in the judge itself. A rubric-based grader can be wrong in ways nobody notices until its verdicts get checked against real labels, which is a separate problem from grading the agent, and the reason a [golden set still matters](/answers/eval-datasets/when-do-i-still-need-a-golden-set) even once you've started.

---

Source: https://tessary.ai/answers/eval-datasets/can-i-start-agent-evals-without-a-labeled-dataset
More on Eval datasets: https://tessary.ai/answers/eval-datasets
From Tessary, agent reliability for AI agents in production: https://tessary.ai