Claude agent SDK evals

The Claude Agent SDK is the harness behind Claude Code, published so you can build your own agent on the same loop: gather context, act, check the result, repeat. It gives you the tool loop, file and shell tools, subagents, MCP support, sessions you can resume or fork, and hooks at fixed points in the loop.

Hooks and subagents are the two features that matter for evaluation. A hook is a callback at a named point, such as before a tool call or after one. That is where a check or a trace emit attaches without touching the agent's prompt. A subagent runs in its own context and returns a result to the main loop, so it is a natural unit to grade on its own instead of through whatever the main agent finally says.

Telemetry is OpenTelemetry, turned on with CLAUDE_CODE_ENABLE_TELEMETRY and pointed with the standard OTEL exporter variables. Metrics and events are the settled part: tokens and cost per session, and an event for each prompt, response, API request, API error, tool result, and tool decision. Spans are still behind a beta flag, and their names are Claude Code's own rather than the OpenTelemetry GenAI ones, so anything reading them has to map them first.