What does a production span carry that a dataset row cannot?

A production span carries context nobody writes into a dataset row by hand: session.id, deployment.environment.name, and the exact tool call, gen_ai.tool.name plus the arguments it actually ran with, not a stand-in for one. A hand-built eval case can approximate a prompt and an expected answer, but it can’t manufacture those fields, because they only exist because a real request happened at a real moment inside a real deployment. That’s what makes production traces catch regressions a static dataset can’t: they carry the conditions a failure happened under, not just the text that was said. It’s also why they’re evidence rather than examples. When something goes wrong, the trace is what a root cause analysis points back to; a dataset row assembled ahead of time was never anchored to an actual event.