What's the difference between concept drift and data drift?

Data drift is the input distribution changing while the correct output for a given input stays the same; concept drift is that relationship itself changing, so the same input now warrants a different answer.

The standard framing: “data drift is the change in the input data distributions; concept drift is the change in relations between model inputs and outputs,” and one can happen without the other, so a model can keep performing reliably through data drift alone. For an agent, data drift looks like a new mix of requests that still get the same right answer they always did. Concept drift looks like the rule itself changing underneath the agent: a refund policy that used to allow 30 days now allows 14, so a request the agent used to correctly approve now needs a correct denial.

Tessary’s behavior_drift classifier sits closer to the data-drift side of that line: it flags a shift in the steps an agent takes without judging whether the new steps are correct, the same limit plain data drift has on its own. Telling a harmless shift apart from a concept that quietly changed is a judgment call it hands to a person.

sources

keep reading

More on this.

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y