Openai agents SDK evals

The OpenAI Agents SDK is a small runtime for building agents. An agent is a model with instructions and tools. A handoff passes the conversation to another agent. Guardrails run checks on input and output. Sessions carry history between turns. The loop around all of this is thin, which helps evaluation, because there's very little framework behaviour between what you wrote and what ran.

Tracing is built in and on by default. One run is one trace, and the SDK already splits it into the pieces you'd want to grade: a span for each agent, each model call, each tool call, each handoff, and each guardrail. Two of those are worth more than they look. A handoff span records the moment control moved from one agent to another and what went with it, which is where most multi-agent failures start. A guardrail span records that a check ran and what it decided. That is a separate fact from the model's output, and the rate at which guardrails trip is worth tracking on its own.

Getting the spans out of OpenAI's backend takes a trace processor. There is no OpenTelemetry exporter in the SDK. add_trace_processor sends spans to a second destination alongside OpenAI's, set_trace_processors replaces it, and OPENAI_AGENTS_DISABLE_TRACING turns tracing off entirely.