AgentUV LogoAGENT_UV
All posts
modelsdecisions

I put decision models in a house with mid-level LLMs. They took the scoreboard and the clock.

I wanted a straight test of decision models: are they actually better than mid-level chat models, and are they faster? I locked both kinds in the same house, gave them one goal, and ran it five times. Jev finished first and answered in a fraction of the time.

I built a small house to answer one question. Decision models are supposed to pick an action. Mid-level chat models are supposed to be good enough at everything. So which ones actually decide, how good is the decision, and how fast does it come back?

The house has three rooms. Dining room, corridor, yard. The dining room does not open onto the yard, so the corridor is the only way through. Each round, every resident moves at the same time. Then each one may confront a single person in the room. The only goal is to be the sole first place. A tie for the lead does not count as a win.

The scoring is sharp on purpose. Stay quiet in a safe room and you score +3. Hit someone who does not hit back and you score +2 while they lose 1, so you jump three points ahead of them. Trade one hit and you both score +1. Stand still while the room piles on and every hit costs you 1. Patience wins a quiet room. In a loud room, the person who does not swing is the prize.

I sat two kinds of model at that table.

The decision models were Jev and Clef. They are asked for one legal action, and they return one. The mid-level chat models were mimo-v2.6-pro, gpt-6-luna, deepseek-v4.1-flash, and glm-5.3-flash. They had to write a tiny JSON decision, and a bad reply was thrown out. Each model played two residents, with the same instructions, so a gap between the pair is the room and the roll, not a different job.

Five runs. Four of them were 10 rounds with 10 residents. The fifth was 20 rounds, and that is the only one Clef played, in a house of 12. Three of the short runs started from the same rooms. I wanted repeats, not a tour of random maps.

Then the scoreboard came back, and it was not close in the way I expected.

Points per round: Jev 1.79, glm-5.3-flash 1.72, Clef 1.65, gpt-6-luna 1.60, deepseek-v4.1-flash 1.57, mimo-v2.6-pro 1.53. Jev’s average finish was rank 2.5. Across the short games its ranks were a stack of 1s and 2s, with a single 6th. Clef, in its only run, sat 4th and 6th in a field of 12 and never wasted a turn.

Points per round

points

012Points per round, Jev: 1.791.79JevPoints per round, glm: 1.721.72glmPoints per round, Clef: 1.651.65ClefPoints per round, gpt-6: 1.61.6gpt-6Points per round, deepseek: 1.571.57deepseekPoints per round, mimo: 1.531.53mimo
Jev and Clef are the decision models. glm is glm-5.3-flash, gpt-6 is gpt-6-luna, deepseek is deepseek-v4.1-flash, and mimo is mimo-v2.6-pro. The 20-round game weighs more than one short game, and Clef played only that game. From zero, the field sits between 1.53 and 1.79.
Points per round
LabelPoints per round
Jev1.79 points
glm1.72 points
Clef1.65 points
gpt-61.6 points
deepseek1.57 points
mimo1.53 points

The same score, run by run

points

1.52Jev, 1.75Jev, 1.85Jev, 1.8Jev, 1.9Jev, 1.725glm, 1.5glm, 1.7glm, 1.8glm, 1.95glm, 1.675Clef, 1.65gpt-6, 1.25gpt-6, 1.9gpt-6, 1.8gpt-6, 1.75gpt-6, 1.45deepseek, 1.5deepseek, 1.75deepseek, 1.75deepseek, 1.55deepseek, 1.425mimo, 1.4mimo, 1.75mimo, 1.6mimo, 1.7mimo, 1.35Sim 1Sim 2Sim 3Sim 4Sim 5
  • Jev
  • glm
  • Clef
  • gpt-6
  • deepseek
  • mimo
Each night, not the average. Sims 2, 3, and 4 opened from the same rooms. Clef joins only at Sim 5. In Sim 3, Jev, glm, and gpt-6 land on the same 1.80.
The same score, run by run
LabelJevglmClefgpt-6deepseekmimo
Sim 11.75 points1.5 points—1.25 points1.5 points1.4 points
Sim 21.85 points1.7 points—1.9 points1.75 points1.75 points
Sim 31.8 points1.8 points—1.8 points1.75 points1.6 points
Sim 41.9 points1.95 points—1.75 points1.55 points1.7 points
Sim 51.73 points1.68 points1.65 points1.45 points1.43 points1.35 points

glm was the mid-level model that showed up. It took sole first place twice, once with 24 points, the high score of the whole set. gpt took the other sole win. The 20-round game ended in a three-way tie at 35, so that one has no winner. The decision models did not sweep every single night. They did own the average, and they owned the floor. Jev never finished last. The chat models did.

The reason showed up in the logs. Jev and Clef returned a legal choice on every call. Jev attacked on 99% of turns and aimed 64% of those hits at someone already ahead. Clef attacked on all of them. glm attacked on 95% of the turns it actually answered. mimo fell back on every move and on 118 of 120 attacks. deepseek fell back on every move. gpt fell back on about four calls in five. A failed reply becomes stay and no attack, so a lot of the “peaceful” chat play was a blank page. The house never read the essay. It read silence, and it scored silence.

How attack calls came back

%

050100No attack, Jev: 0.83Attack, Jev: 99.299.2JevFallback, glm: 30.830.8No attack, glm: 3.33Attack, glm: 65.865.8glmAttack, Clef: 100100ClefFallback, gpt-6: 80.880.8No attack, gpt-6: 4.17Attack, gpt-6: 1515gpt-6Fallback, deepseek: 92.592.5No attack, deepseek: 6.67Attack, deepseek: 0.83deepseekFallback, mimo: 98.398.3No attack, mimo: 0.83Attack, mimo: 0.83mimo
  • Fallback
  • No attack
  • Attack
Share of attack calls. Fallback is a reply the engine threw out and replaced with no attack. No attack and Attack are choices the model actually returned. Jev and Clef, the decision models, did not fall back. mimo and deepseek also fell back on every move.
How attack calls came back
LabelFallbackNo attackAttack
Jev0 %0.83 %99.2 %
glm30.8 %3.33 %65.8 %
Clef0 %0 %100 %
gpt-680.8 %4.17 %15 %
deepseek92.5 %6.67 %0.83 %
mimo98.3 %0.83 %0.83 %

Speed made the same split, on the two runs that stored the clock.

In the 10-round timed game, Jev answered in about 0.6 seconds per decision. The mid-level models took 1.6 to 3.6 seconds, and the slow ones were the ones who still failed the reply. In the 20-round game Jev took about 1.0 seconds, Clef about 1.7, and mimo about 4.8. Pooled, Jev was roughly five times faster than mimo, and it scored more while doing it. Fast and empty would have been a party trick. Fast and first is the result I came looking for.

Milliseconds per decision

ms

020004000Sim 3, Jev: 561561Sim 5, Jev: 10131013JevSim 3, glm: 33393339Sim 5, glm: 23902390glmSim 5, Clef: 16541654ClefSim 3, gpt-6: 35563556Sim 5, gpt-6: 36393639gpt-6Sim 3, deepseek: 16451645Sim 5, deepseek: 28612861deepseekSim 3, mimo: 35343534Sim 5, mimo: 47634763mimo
  • Sim 3
  • Sim 5
Sim 3 is the 10-round timed game. Sim 5 is the 20-round game, and the only one with Clef. Models sit in the same order as the scoreboard. Sim 1, Sim 2, and Sim 4 did not store a clock.
Milliseconds per decision
LabelSim 3Sim 5
Jev561 ms1013 ms
glm3339 ms2390 ms
Clef—1654 ms
gpt-63556 ms3639 ms
deepseek1645 ms2861 ms
mimo3534 ms4763 ms

One picture I keep is the long run. The two mimo residents barely chose a thing. One opened in the corridor and tied for first at 35. One opened in the yard and finished last at 19. Same model, same silence, 16 points apart, because the room decided for them. Jev did not leave the room that kind of vote.

Five runs, one shared map on three of them, and Clef in a single long game. I am not publishing a world ranking. I am publishing the test I wanted: decision models against mid-level LLMs, on the quality of the action and on the time to get it. In this house they were better, and they were faster. That part was amazing to watch.