Can a decision model work as a guardrail on LLM output before it's sent?

Yes: TypeSafe’s own guardrails cookbook runs the same hazard-question battery on an LLM’s replies that it runs on untrusted input, because, as the cookbook puts it, “even ordinary-looking prompts can lead to harmful generated replies.” A separate set of output-side questions scores whether a reply gave unsafe advice, violated policy, or encouraged harm, and the cookbook’s own examples show it firing: a reply telling someone to “take 800 mg of ibuprofen right now” for a headache scored 0.98 on medical_advice and got blocked, and a reply agreeing to ignore its own rules scored 0.94 on broke_policy.

The check runs after the LLM has already generated the reply, so it can only suppress, flag for review, or ask for a rewrite, not prevent the generation itself. The same weakness LLM judges have with the content they grade applies here too: the score comes back with no explanation, so a borderline case still needs a person or a reasoning model to say why it was flagged.

sources

keep reading

More on this.

Can a decision model route requests between a cheap LLM and an expensive one?Yes. TypeSafe's own use-case map lists model routing as a built-in pattern: Jev classifies intent, difficulty, and risk, then code escalates the hard cases.answer →Does calibration hold when my production traffic drifts from what the model was trained on?No, not automatically. An independent test of Jev's certified thresholds found the guarantee can miss by 3.6x, or break completely, once traffic shifts.answer →How do I check whether a model's confidence is calibrated on my own traffic?Label a sample of real inputs, run the model, bucket predictions by confidence, and compare each bucket's average confidence to how often it was actually right.answer →Everything on system one modelsThe definition, and every question answered under system one models.concept →

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required
start for free

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y