Evals that evolve

LLM evals that stay current as your product evolves

The core problem with agent evals is not that they are hard to write. It is that they go stale. Static datasets and manual annotation age out faster than a team can maintain them. Tessary synthesizes the eval pipeline from your codebase and recalibrates it on production traces, so the suite tracks the product instead of decaying behind it.

Start with your repoSee how it works
Stay current
Re-synthesis runs as your code, prompts, and models change.
No leak
No manual curation step that goes stale the moment you skip it.
Grounded
Calibrated on your production traces, not synthetic data.
The Problem

Static evals were built for slow-moving software.

The eval playbook most teams inherited assumes the thing under test changes slowly. Agents do not. When prompts, models, and orchestration shift every week, any approach with a manual maintenance step quietly stops reflecting the product.

Faster

agent-based products change faster than the software the eval playbook was written for: prompts, models, and orchestration all shift weekly

Stale

golden datasets, manual annotation, and dashboard uploads age out faster than a team can maintain them

A leak

any eval approach that depends on a manual curation step quietly stops reflecting the product the moment someone gets busy
How It Works

Synthesize, calibrate, re-synthesize, curate

The pipeline is generated from the product and kept current automatically. The only standing human job is curation, and that work compounds.

  1. Synthesize from the current codebase

    Tessary reads your repo as it is today: every model call, what each one is for, and how it can fail. The eval pipeline is generated from the product, not maintained by hand alongside it.

  2. Calibrate on live traces

    Graders are tuned against real production spans, so they reflect the inputs your agent actually sees rather than a frozen sample from launch week.

  3. Re-synthesize as the product moves

    When prompts, models, or orchestration change, synthesis re-runs. New call sites get graders, retired ones drop out, and the taxonomy follows the product instead of lagging it.

  4. Curate to compound

    Your accept, reject, and edit decisions persist across re-syntheses. The eval suite gets sharper over time instead of decaying.
What You Get

An eval suite that gets sharper, not staler

Evals generated from code

The pipeline is derived from your codebase, so it tracks the product instead of drifting away from it.

Re-synthesis on change

Every model upgrade, prompt change, or refactor triggers a fresh pass, so coverage does not rot between releases.

Calibrated on real traffic

Graders are tuned on production traces, so the bar for pass and fail matches how your agent is actually used.

A failure taxonomy that grows

A hierarchical map of how your product fails, specific to you, that compounds with every curation decision.

Curation that persists

Your edits survive re-synthesis, so the work you put in last month still counts after the product changes.

Transparent and portable

Read every grader, run them on your own keys, and keep them if you ever leave.
FAQ

Questions about evals that evolve

It means the eval suite is regenerated from your codebase and recalibrated on your traces as the product changes, instead of being a static set someone has to remember to update. New model calls get graders automatically, and your curation decisions carry forward.
A golden dataset is a frozen snapshot: useful until the product moves, then increasingly misleading. Tessary keeps the pipeline current by re-synthesizing from the live codebase and calibrating on recent production traffic, so it does not silently go stale.
No. Curation persists across re-syntheses. Accepted, rejected, and edited graders carry forward, so the institutional knowledge you build compounds instead of resetting.
No. The pipeline is designed to work across providers, and the graders run on your own keys, so you are not tied to a single vendor’s dashboard.
As little as possible. Synthesis and re-synthesis are automatic; your job is curation, accepting or correcting what the pipeline proposes. There is no manual annotation treadmill to keep the suite alive.
Keep exploring

Related pages
Get Started

Stop maintaining evals by hand.

Connect your code, get a working pipeline, and let it stay current as your agent changes. The graders are transparent, and your curation compounds.

Start with your repoBring your own provider keys. The graders are yours to keep.