# What's the difference between an LLM judge and a reward model?

An LLM judge is a general model, prompted at grading time with the criteria you want checked; a reward model is a separate model trained specifically to output one preference score, with the criteria baked into its training data rather than a prompt.

Change what an LLM judge should check and you edit the prompt; change what a reward model should check and you retrain it on new preference data, which is slower and needs a labeled dataset to start. A reward model also returns a single number with no explanation, while an LLM judge can be asked for a verdict plus the reasoning behind it, so its calls can be read and audited one at a time rather than trusted as a black box. Reward models are the common choice inside training, like RLHF, where the same scoring policy has to run millions of times cheaply; an LLM judge is the more common choice for evaluation, where criteria change often. [A judge's per-event cost is why it usually runs on a sample rather than every trace](/answers/eval-costs/why-does-a-classifier-cost-less-to-run-than-an-llm-judge), a tradeoff a reward model's cheap, fixed inference doesn't face the same way.

---

Source: https://tessary.ai/answers/llm-as-judge/whats-the-difference-between-an-llm-judge-and-a-reward-model
More on LLM as judge: https://tessary.ai/answers/llm-as-judge
From Tessary, agent reliability for AI agents in production: https://tessary.ai