How do I calibrate a CI gate before I let it block a merge?

Run it in report-only mode first and watch what it would have blocked before you let it block anything. Tessary’s CI gate ships with risk-threshold at 1.0, which disables the risk trigger outright, and block-on-regression at false, so a fresh install posts its verdict on every pull request without stopping a merge. Once you’ve watched it against enough real PRs to trust the forecast, lower the threshold in your project config so it starts blocking.

You don’t have to commit to a number blind, either. A single request to the gate’s report endpoint can pass a gate_threshold that overrides the config for that one PR, so you can try a stricter bar on one change and see what it would have caught before you redeploy the config for everyone. The gate blocks on two triggers, OR’d: predicted risk clearing your threshold, or a grader measurably regressing once block-on-regression is on.

keep reading

More on this.

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y