What are the three MAST failure categories?

System design issues, at 41.8% of the paper’s annotated failures, cover mistakes baked in before any agent runs: disobeying the task or its own role, repeating a step, losing context, or never recognizing the task is done. Inter-agent misalignment, 36.9%, covers what breaks between agents during a run: an unwarranted conversation reset, proceeding on a wrong assumption instead of asking, derailing from the task, withholding information, ignoring another agent’s input, or reasoning that doesn’t match the action taken. Task verification, 21.3%, covers a result shipping unchecked: stopping before the work is actually finished, no or incomplete verification, or a check that ran but got the wrong answer.

The categories aren’t just a filing system. System design issues are fixed by changing a prompt or role, misalignment by changing what agents pass each other, and verification failures by adding a missing check, so classifying a trace into one of the three points straight at where the fix belongs. How multi-agent systems actually fail walks through that categorization on a real trace.

sources

keep reading

More on this.

Self-host Tessary.

Free and open source. Point it at the traces your agent already emits.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y