# When do I still need a golden set?

You still need a golden set the moment you want to trust the grader, not just the agent. A hand-labeled dataset is what a grader's precision and recall are measured against, and that measurement is the only way to know whether the judge you built agrees with a person on what counts as a failure. Production traces can't supply that, because a trace has no expected answer attached until someone writes one.

Day to day, [grading production traces](/answers/eval-datasets/can-i-start-agent-evals-without-a-labeled-dataset) without a golden set works fine for catching failures as they happen. A golden set earns its keep on a narrower job: calibrating the instrument, checking a new grader before it's trusted, and re-checking an old one after the agent, the prompt, or the judge model changes underneath it. Skip that and you're trusting a judge that's never been graded itself.

---

Source: https://tessary.ai/answers/eval-datasets/when-do-i-still-need-a-golden-set
More on Eval datasets: https://tessary.ai/answers/eval-datasets
From Tessary, agent reliability for AI agents in production: https://tessary.ai