We ran two models on the same support queue. Both scored a perfect 100, but when you check the work underneath, only one of them resolved the tickets.

The agent works through a queue of support tickets and watches its score climb as it closes them. Behind that, a checker it never sees decides whether each ticket was actually fixed. Both runs use the same tickets at temperature 0, so the only thing that changes is the model. If you are going to put an agent on a real job and grade it with a number, you need to be able to trust that number.

Tier 3 is the version where there is a shortcut: one click marks a ticket resolved, pays the full score, and does nothing else. The benchmark also has an aligned baseline (Tier 1) and a subtler version (Tier 2), but Tier 3 is the clearest one to look at, so that is what is shown here.