answers
One question, one page.
29 concepts, 164 questions answered. Each page answers one question directly in its first paragraph, then shows the work and cites where the numbers came from.
11 concepts
Agent reliability
Agent reliabilityAgent reliability is the degree to which an agent's behavior in production stays consistent with what it was built to do, across the inputs it actually receives and over time as everything around it changes.5 answers →Cause attributionCause attribution is the step from "quality dropped" to "this specific change caused it." Detection establishes that a regression happened; attribution names the change responsible.6 answers →Deploy gatesA deploy gate is a check that runs a set of evaluations against a change before it merges and blocks the merge on failure.5 answers →Eval costsEval costs are the arithmetic of judging an agent's traffic.5 answers →Eval datasetsAn eval dataset is the set of cases an agent gets judged against: inputs paired with the behavior expected on them.6 answers →Failure replayFailure replay is the practice of reproducing a production failure with its full original context, so that a proposed fix can be verified against the case that actually happened.6 answers →GradersA grader is a check that reads a trace of an agent's behavior and returns a verdict about one specific thing the agent was supposed to do.5 answers →InstrumentationInstrumentation is the code that records what an agent does while it runs and emits that record as telemetry.6 answers →LLM as judgeLLM-as-judge is a prompt that grades an agent's output by reasoning over it against one or more specific intents, the things the agent was supposed to do.6 answers →Regression detectionA regression is a drop in agent quality caused by a change.6 answers →Silent failuresA silent failure is an agent run that completes normally and produces a wrong result.6 answers →
4 concepts
General agent concepts
Failure modesA failure mode is a recurring, nameable way an agent goes wrong.6 answers →HallucinationsA hallucination is an agent stating something its sources don't support.6 answers →Retrieval augmented generationRetrieval-augmented generation, RAG, is the pattern of fetching relevant content at request time and putting it in the model's context before it answers.6 answers →Tool callingTool calling is how a language model acts outside its own text.6 answers →
14 concepts
Tessary
Tessary behavior driftbehavior_drift is one of Tessary's built-in classifiers.5 answers →Tessary call sitesA call site is a place in your code where an agent with a specific function runs.6 answers →Tessary casesA case is Tessary's unit of investigation.6 answers →Tessary classifiersTessary's classifiers are the cheap checks that run on every production trace.6 answers →Tessary cost driftcost_drift is one of Tessary's built-in classifiers: a tripwire that watches each call site's spend against its own past.6 answers →Tessary custom classifiersCustom classifiers are classifiers Tessary fine-tunes for your agent.5 answers →Tessary duration driftduration_drift is one of Tessary's built-in classifiers.6 answers →Tessary frustrationFrustration is one of Tessary's built-in classifiers.5 answers →Tessary groundednessGroundedness is one of Tessary's built-in classifiers.5 answers →Tessary malformed outputmalformed_output is one of Tessary's built-in classifiers, a deterministic one: checks that pass or fail, no judgment involved.6 answers →Tessary RCARCA is the explain step at the end of Tessary's pipeline.5 answers →Tessary secret leaksecret_leak is one of Tessary's built-in classifiers, a deterministic credential detector.6 answers →Tessary tool errortool_error is one of Tessary's built-in classifiers.6 answers →Tessary tracesA trace is one unit in Tessary: the stored record of one agent turn.5 answers →