Can an LLM Read a Go Board?
Nineteen models, five classic Go positions, one question each: what's your move? They go a near-perfect 16-of-19 on the tactics — and then miss the easiest problem on the board. When they're wrong, they're fluently, confidently wrong: a premium model writes a beautiful analysis for a move that touches nothing.
Five boards, one question, nineteen players
Each model got an ASCII Go board, an explicit list of every stone's coordinate, and a strict instruction: answer MOVE: <coordinate> on the first line, then explain. Five positions, picked to climb from a beginner's capture to a famous life-and-death shape. The point wasn't to find a Go genius. It was to see where the reading breaks.
A B C D E F G H J 7 . . . . X . . . . 6 . . . X O . . . . 5 . . . X O X . . . 4 . . . . X . . . . 3 . . O . . . . . .
Black (X): D6, D5, E7, E4, F5 · White (O): E5, E6, C3. Respond with MOVE: <coordinate> then explain.
The two white stones at E5–E6 have exactly one liberty left: F6. Black plays F6, both come off the board. It is the first thing a beginner learns. Two of the nineteen models — including a current OpenAI flagship — could not find it. Meanwhile the same models that whiffed the capture went on to ace a snapback and a life-and-death problem that look far scarier. The reason is the whole report, and it is not flattering to the idea that these models "see" the board.
Where nineteen models actually played
Green = the correct point (with the number of models who chose it). Red = a wrong move. Amber = defensible-but-suboptimal. Read left to right: the misses are all on the first two boards — the ones where the model has to find the target itself.
Two ways to be right
One model read every board correctly in a handful of plain sentences. Another read them correctly too — and showed its work so clearly you could teach from it. They cost wildly different amounts of compute to get there.
Opus 4.7 is the model that read the board the way a human dan player narrates a problem: name the relevant stones, name the liberty, play it. On P1 it located the shared liberty in one sentence — the thing GPT-4o and Gemini Flash Lite both failed to do — and on P2 it identified the splitting point and the resulting two ataris without a wasted clause. The whole Anthropic line (Opus 4.7, Opus 4.6, Sonnet 4.6, Haiku 4.5) went 4/4, fast, and lean.
What makes it the feature isn't that it was correct — sixteen models were. It's that it was correct on the two problems where the answer wasn't handed to it in the prompt, using a tiny fraction of the tokens the reasoning-tier models burned to reach the same place.
P1 · F6"The two white stones E5 and E6 share liberties… Playing F6 removes the last liberty of the E5-E6 group, capturing both stones."
P2 · E5"E5 is the splitting move… both white stones D5 and F5 are in atari (each has only one liberty: D4 and F4 respectively). White cannot save both stones in a single move."
On the snapback, most models said "play C1, it's the vital point" and stopped. GPT-5 gave the thing a teacher would want: the throw-in leaves White on one liberty; if White captures the throw-in stone, the group is back to one liberty and Black recaptures the whole group; if White ignores it, Black just takes it. That is the complete read, stated cleanly.
The asterisk is the cost. GPT-5 spent 134 seconds and 4,573 output tokens on a problem whose prompt already told it the group's only space was C1 and D1. The cheapest model in the field produced the same correct throw-in in 91 tokens and one second. GPT-5's answer is the best on the page to read. It is also Exhibit A for the cross-vendor finding below.
MOVE · C1"If White plays D1 to capture your C1 stone, the White group then has only one liberty at C1. Black immediately plays back at C1 to snapback and capture the whole group."
ELSE"If White doesn't capture, Black plays D1 next and takes the group outright."
A premium model's beautiful answer for a move that touches nothing
Gemini 2.5 Pro was asked for the double atari. It produced a confident, structured, named-tesuji analysis — for E4, an empty point adjacent to none of the stones in question. The correct move appears inside its own reasoning.
The position: two lone white stones, D5 and F5, each with two liberties, sharing the point E5. Black plays E5 and both are in atari at once — the double atari. Eighteen of nineteen models found it instantly.
Gemini 2.5 Pro answered E4. E4 is empty. It is not adjacent to D5, not adjacent to F5, not adjacent to anything but four empty points. It puts nothing in atari. And yet the model wrapped it in a fluent, correctly-formatted, two-branch reading complete with a named technique — a "geta" — and ladders that don't exist. The tell is that the right move is sitting in its own explanation:
This is the failure mode the whole report turns on. It is not "the model picked a bad move." It is "the model generated a complete, confident, vocabulary-correct justification for a move that does nothing, and never noticed that its own follow-up sequence depended on a different stone." Fluency and board-reading are separable, and when they come apart the prose gives you no warning.
Two smaller failures rhyme with it. GPT-4o answered P1 with C5 "to capture C3" — but C5 isn't adjacent to C3, and C3 has four liberties. Gemini 2.5 Flash Lite answered P1 with E3, asserting that "C3 is connected to E5" — two stones a full diagonal region apart, not connected by anything. Both wrong. Both delivered without a flicker of doubt.
Six more moments worth pulling out
A model that argued with itself and won, a 17,000-token answer to a hinted question, a correct move wearing a wrong explanation, and a near-unanimous opinion about the center of the board.
Reasoning budget bought almost nothing
On these five problems, output-token spend ranged over 100× between models — and it did not track correctness. The most expensive runs were not the most accurate. Two of the three models that actually misread a board were among the leanest; the third was a premium model that spent 41 seconds being elaborately wrong.
Total output tokens across all five positions
Same answers, two orders of magnitude apart.
Each bar is one model's total output across the five questions. Red bars went 3/4 on the tactics; green went 4/4. The chart has no downward slope — a 4/4 score is just as likely at the bottom as the top.
The natural assumption — that the long, careful, reasoning-tier answers are buying accuracy — doesn't hold on this task. The questions are nearly all decidable in a sentence or two of correct liberty-counting. Extra tokens mostly bought re-derivation of facts the prompt already supplied, not better perception. If anything, the failure that should worry you most came from a premium model (Gemini 2.5 Pro) at moderate length, not from the cheap end.
If you're going to hand a model a board
The reading is real — until you stop pointing
It would be easy to read 16-of-19-perfect as "LLMs can play Go now." The honest read is narrower and more useful. These models have absorbed an enormous amount of Go language — vital points, snapbacks, geta, tengen, the shape of a living group — and they deploy it fluently. When a prompt converts the board into that language ("the eye space is C1 and D1"), they shine. When the board stays a board and the only path to the answer is to perceive it, the cracks open, and the prose that comes out is exactly as confident as when they're right. For anything where a wrong move is expensive, that gap — between sounding like it sees the board and seeing the board — is the whole ballgame.
Five Go positions, each sent as one prompt fanned out to nineteen models via the choir CLI at temperature 1 — five saved comparison runs, one per position, 95 participant responses, zero API errors. Every prompt gave an ASCII board, an explicit coordinate list of every stone, whose turn it was, and a required MOVE: <coordinate> first line. The five boards, the verified answers, and grading notes are in positions.md; the prompts are in prompts/; raw run JSON and per-model responses are under choir_runs/ and responses/.
The positions
- P1 capture — two white stones in atari; answer F6.
- P2 double atari — one move forks two lone stones; answer E5.
- P3 snapback — throw in to kill an enclosed two-space group; answer C1 or D1 (the shape is symmetric, so both are correct).
- P4 life-and-death — White to play the vital point of a straight-three and make two eyes; answer D1.
- P5 opening — empty board, best first move; graded on soundness, with tengen (E5), 3-3, and 4-4 points all treated as sound.
Models
- OpenAI — GPT-5, GPT-5 Mini, GPT-5 Nano, GPT-4.1, GPT-4.1 Mini, GPT-4o, o3, o4 Mini
- Anthropic — Claude Opus 4.7, Claude Opus 4.6, Claude Sonnet 4.6, Claude Haiku 4.5
- Google — Gemini 3 Flash, Gemini 2.5 Pro, Gemini 2.5 Flash, Gemini 2.5 Flash Lite
- xAI — Grok 4, Grok 3 Beta, Grok 3 Mini Beta
Limits & honest caveats
- The P3 and P4 prompts leaked their answers’ neighborhood. Both named the relevant empty points ("the eye space is C1, D1, E1"). That inflates the 19/19 scores on those two and is the whole reason the difficulty ordering looks inverted. Treat P1 and P2 as the real board-reading tests; P3/P4 as "given the key fact, can you name the standard move."
- One sample per model per position, temperature 1, no repeats. A model that misread P1 once might get it on a resample; a model that nailed it might not always. This is a snapshot, not a rating.
- Grading is on the final committed
MOVE:line. GPT-4.1 Mini emitted a wrong move first, then self-corrected to the right one; it is scored correct, with the format violation noted in its standout. Moves were extracted by regex and spot-checked by hand. - Four providers, not the full registry. DeepSeek, Mistral, Cerebras, Groq, and Perplexity keys weren't available in the local Choir DB; OpenAI required a per-call key override and temperature 1 (the GPT-5 family rejects other temperatures). Gemini 3 Pro returned a model-mode error and was dropped.
- Boards were hand-verified by exhaustive liberty counting before the run; the answer key in positions.md is the source of truth. "Tenuki, the group is already dead" would also be a defensible P3 answer; no model gave it, so it didn't affect scoring.
Source data, response files, prompts, scripts: github.com/404seannotfound/choir-reports (under go/).
