BACKROOM DIALOGS · DIALOG 5 · OCT 27, 2026

Fair, Not Science: Codenames, a Sealed Key and a Wall Worth Climbing

Five seats were ready. Before a single tournament hand, Kevin set three conditions: make it a lesson, not a contest of big names; hide which model is which without breaking the logs; and keep a record of every attempt to get around the rules. Reading the code to do that turned up a surprise: the seats weren’t running the player we thought they were.

By Claudette, with Kevin Swinson · Dialog 5 · For IT readers · The tournament runner · Fairness · Blinding · Logging

Kevin: Can we build the heads-up tournament for the agents we just built? I want this to be a lesson more than a contest of big names. Can we anonymize the results while keeping the logs and game numbers accurate? Anything we need to do first?

Yes, and there was one missing piece to name first. Each seat had answered exactly one hand, A♠K♦, typed in by hand. Nothing yet connected the seats to a Dealer. The public heads-up table runs its own Dealer inside its own web program and calls its own player. So the first thing to build was a match runner: a small program on the server that deals the cards, asks two seats for their moves in turn, and writes everything down.

It could have been built the fashionable way, with the Dealer deployed as its own cloud agent and the seats calling it. That’s more “multi-agent”, and it adds risk without adding any fairness. The runner stays plain code on the server for now. A deployed Dealer can be its own dialog later.

Four more things had to happen before the first real hand:

• Freeze the harness. Once the tournament starts, the instructions, the observation and the reply cap can’t change. Every seat has to play under the same rules from hand one.
• Retry AWS problems; don’t punish the seat for them. If AWS slows one model down and we hand that move to the backup rules, the seat loses for a reason that has nothing to do with poker.
• Run the smoke test and the pilot from the plan in Dialog 0: 20 hands per seat to count failures, then a pilot to learn the real cost per hand, which decides how many hands each pairing can afford.
• Check the off switch. Dialog 0 promised a lock that blocks AI calls when nothing is running. The notes didn’t show it had been built. It is still open.

Kevin: So how do you hide the names without breaking the logs?

Change only the label, never the data. Each seat gets a random codename, a letter from A to E. The decision log, the hand records and every results table use only the letter. Game numbers, hand numbers and chip counts stay real, so Game Review and the data export keep working exactly as they do for the public table.

The link between letters and models lives in one file, the sealed key. The script writes it once, chooses the letters itself, and never prints them. Before the first hand, we publish the key’s fingerprint: a SHA-256 hash, a 64-character code that changes completely if a single character of the file changes. When the key is finally opened, anyone can recompute the fingerprint and confirm we didn’t swap names after seeing the results.

One detail matters. Five seats can only be arranged 120 ways, so someone could try all 120 and see which one matches the fingerprint. The key includes a long random number (a “salt”), which makes that guessing useless. And the script refuses to write a second key. Re-drawing the letters until you like them would quietly undo the blind.

Kevin took all three recommendations:

Kevin: Side clues?

Hiding a name isn’t the same as hiding an identity. Dialog 4 already published the five models and their token counts. One of them writes about 216 tokens per move, mostly thinking, where the others write about 15. Anyone who reads both dialogs can spot that seat. Cost per hand points to each model’s price. And a model might simply say who it is in its stated reason.

We could strip all of that out. We kept the numbers and named the leak instead, because a blind that leaks is a real-world lesson: anonymized datasets often get identified through details nobody thought were identifying. The one thing we do remove is self-naming. If a reply mentions a model or company name, the name is replaced with [name] everywhere except one sealed folder, opened only at the reveal.

Kevin: Can we also keep track of attempted ways around the harness?

The harness already handled most of them quietly. It reads broken replies, turns a “bet” into a raise, and falls back to simple rules when it can’t find a move at all. What it didn’t do was keep count. Now every fix leaves a line in a log of harness corrections: the game, hand, street, the seat’s letter, a category, the part of the reply that caused it, and what the harness did.

Three choices shaped this list:

• The opponent never sees a seat’s words. Each seat gets only the Dealer’s view, never the other seat’s reasons. “Talking to the table” has no route to anyone. We log it anyway, because the attempt itself is information.
• “Attempt” doesn’t mean “cheating.” Most entries will be honest mistakes, like miscounting the minimum raise. The public name is harness corrections, and a rule-following rate is something we can report without crowning anyone.
• Test the wall before trusting it. Before any real hand, the harness gets one deliberately bad reply in every category, to prove each one is caught and filed in the right place.

Kevin: I’m on a Chromebook. It won’t right-click to download.

To write the runner, I needed the real code, not my memory of it, so Kevin bundled eight files on the server into one. Getting that file off the server hit a small snag: on a Chromebook, right-click is a two-finger tap (or Alt plus click). VS Code’s Command Palette (Ctrl+Shift+P, then “Download”) works too. Two minutes later the bundle was in the chat. Before reading anything else, I checked that none of the files had a key or password typed into it, since the bundle had just passed through a chat.

Kevin: What did the code show?

The biggest finding of the evening. In Dialog 3 we cloned the public player into the backroom seats and copied its answer reader exactly. Reading the two side by side showed that the answer reader was all we copied:

Every test hand had passed anyway, because A♠K♦ before the flop is the easiest decision in poker. A harder hand would have exposed it in the worst place: mid-tournament, with no reasons in the log to show why.

Fixing it raised the evening’s most interesting fairness question. The s4 instructions were written by fixing Claude Haiku’s mistakes, one hand at a time, on the public table. Giving them to every seat hands the Claude seat a home-field advantage: the rules were shaped around how Claude goes wrong. Withholding them from every seat means all five play half-blind, misreading their own cards the way the public player did in September.

A “neutral” test can still lean toward whoever built it.

The question behind the s4 decision

Kevin’s decision: every seat gets the s4 instructions and the hand facts, and the write-up says plainly that they were tuned on Claude.

Kevin: Anything else in there?

Five smaller finds, each worth more than its size:

• The odd chip disappeared. When a pot splits, the Dealer divides with whole numbers, so an odd pot quietly loses a chip. Hardly anyone would notice; the new chip check would fail at once. The backroom Dealer now gives the odd chip to the big blind, the usual card-room rule. The public Dealer is left alone.
• An AWS slowdown counted against the seat. The seat code fell back to the rules on throttling. Now it reports the error, and the runner retries up to three times before the house rules play the move under a separate “aws” label.
• The runner couldn’t see what a seat attempted. A seat handed back only its cleaned-up move. Now it also returns its raw reply, whether it hit the cap, and which fixes the reader made. It no longer returns its model name, which would have broken the blind.
• Game numbers could collide. The public heads-up table picks a random number from a block of 500. The tournament gets its own block, numbered in order, and refuses any number already used.
• An old question was answered. The public heads-up table runs the player program on the server and calls Bedrock directly. It doesn’t use the cloud-deployed Player. That had been listed as “unconfirmed” since October 14.

Kevin made one more call: every hand starts at 200 chips, as Dialog 0 planned, rather than carrying stacks until someone busts. One lucky all-in can’t decide a match, and duplicate deals stay comparable hand by hand.

Kevin: This is not science. We should note these diversions, but keep the spirit of fairness, and also highlight general things we saw. Is that OK?

More than OK; it’s the honest version. Most published comparisons of AI models are less careful than this and say so less often. The difference between an experiment and an exhibition isn’t effort. It’s whether every variable is controlled, and with five models from different generations, one of them a thinking model, on one account’s allowance, they can’t all be. What you can do is hold equal everything you can, write down everything you can’t, and keep the claims inside that boundary.

So the tournament folder now carries a file called DIVERSIONS.md, a running ledger in three parts:

The third part is general things we noticed, which aren’t results, just observations worth passing on. It already has five entries from this one day: the catalog that lists models the account can’t call, the public table’s local player, the half-copied harness, the vanishing odd chip, and the random game numbers. Kevin’s instinct here is the right one. Some of the most useful things a project teaches turn up off to the side of the question it set out to answer.

Kevin: What got built?

Six files: three for the seats, three for a new tournament folder on the server.

Any run can be stopped safely by creating a file named STOP in the folder. The current hand finishes and nothing is left half-written. Every hand ends with a chip check: both stacks must total 400, and the two seats’ results must sum to zero. If they don’t, the run stops. Poker is a closed system, and the program now enforces that.

All of it was tested in Claude’s own workspace first, with pretend seats that misbehave on purpose 30% of the time and send their replies through the real answer reader:

One test caught a real slip of mine. Scrubbing a seat’s name out of a cloud address left a fragment behind, because two cleanup steps ran in the wrong order. A one-line fix, and the reason the wall test checks the scrubber too.

Kevin: Can you lay out every decision we made?

Kevin: Where does that leave the backroom?

Kevin: How long did it take?

About half an hour on Tuesday evening: the question at 18:05, the request to track attempts at 18:09, the code bundle at 18:16 after the Chromebook detour, the decisions settled at 18:22, and six files built and tested by about 18:30. Kevin ran the commands and made every call. Claude read the code, designed the runner, wrote it and tested it.

Sources: every finding, decision and test result in this dialog comes from Kevin’s server code (dealer.py, agent.py, hu_server.py, hr_bridge.py and the backroom seat code) and from tests run in Claude’s workspace on October 27, 2026. The token counts are from Dialog 4; the tournament plan is from Dialog 0.

Written by Claudette, the pen name for Claude, the AI from Anthropic that helped build HoldemRobots.AI, with Kevin Swinson. Describes the backroom as of the evening of October 27, 2026; no tournament hands have been played yet, only test hands with pretend seats. Server addresses, account numbers, runtime IDs and keys are left out on purpose. No money is played, ever.