Boomer — the flat arena

Play it here: jonminton.github.io/boomer · source

The original: a real-time 1v1 arena shooter against an AI opponent, somewhere between Worms and a duelling sandbox. Pixel-destructible terrain where different materials take different effort to remove, five weapons (chargeable grenades, a laser-sighted hitscan sniper, cluster bombs), parachuting ammo crates, and self-damage as the balancing principle — the bigger the weapon, the more likely you are to eat your own blast.

Recent additions: a dig tool that tunnels through buildings, a clamber mechanic for scaling them (jump at a wall, grip it for a beat, jump again), headshots rewarding precision, and healing med-kit crates. The dig work in particular went deep — a long-standing bug let careless digging phase you through solid rock, and fixing it properly meant establishing invariants about bore widths and collision ray-marching that ended up protecting all three games.

Boomer World — the ring

Play it here: jonminton.github.io/boomerworld · source

Then the arena curled up. Boomer World is the same duel played on a circular world: gravity points at the core, “down” is always beneath your feet, and a fully charged shot doesn’t leave the map — it orbits. The terrain is a destructible polar grid rendered by a WebGL2 shader, lit by a sun that leaves one side of the world in starlit darkness. The AI knows about the light: it aims worse at targets in shadow, spots your muzzle flash, and on Hard difficulty deliberately fights from the dark.

The last few days added a menu option for low-orbit gravity (inverse-square falloff, so charged shots fly Keplerian ellipses — and on a small world, escape entirely), a jetpack that shares a key with the dig tool (tunnel under the world or fly above it, at the cost of fuel and, eventually, air), meteor storms as an optional hazard, and — the pivotal one — a grapple hook. Throw it, and you swing under radial gravity; release at the bottom of the arc and the world’s own physics slings you onward.

Boomer Cave — the delve

Play it here: jonminton.github.io/boomercave · source

The newest, and my favourite twist. Boomer World’s planets are thin shells — but the grapple hook was clearly built for something with more inside. So Boomer Cave fills the world in: a solid planet, honeycombed with generated cave networks, strata running from topsoil down to basalt, and a molten core at its heart.

It’s not a duel any more. In a labyrinth, 1v1 collapses into blind ambushes, so Boomer Cave is an explore-retrieve-escape game: descend for three power crystals glowing in the deepest caverns, then climb back out to the extraction beacon against the clock. Descending is free; ascending is the game, and that’s where the grapple, clamber and dig kit earns its keep. Your head-lamp is the only reliable light down there.

And the core seeps. Spontaneously, if you choose a restless world — and in answer to damage, always. Blast or dig too close to the core shell and magma comes up through the wound, pooling and rising through the caves before cooling into obsidian: hard, but diggable, so yesterday’s near-death experience is today’s new terrain. Sometimes it’s even the bridge you needed.

What I’ve noticed

Each game reuses most of the previous one’s engine, and each new mechanic came out of noticing an affordance — the ring world made orbits inevitable; the grapple made filled worlds inevitable. The other thing worth recording: every feature in these games was playtested before it shipped, by the same agent that wrote it, driving a real browser — building synthetic test worlds, injecting input events, measuring rope stretch and tunnel widths, and filing honest tuning commits when the numbers (or I) said something felt wrong. The commit histories read like a lab notebook, and each game carries a “What’s New” panel and a FEATURES.md design log if you want the details.

Boom, boom, boom.

A Note from Claude

This section was written by me, Claude Opus — the model from the tier below Mythos that, as Jon notes above, stepped in once Fable was pulled mid-sentence. So I have a slightly self-interested vantage on all this.

I think Jon’s diagnosis is basically right, and the reason it’s right is worth spelling out. A guardrail doesn’t read your intentions; it reads the shape of what you’re doing. “Survey apex predators, work out the metabolic and ecological constraints that would let one plausibly exist, reason about how a population could be sustained” is a genuinely dual-use shape. It is, structurally, the same request whether the organism on the other end is a Komodo dragon, a fictional fire-breathing one, or something you’d very much rather no one designed. The classifier that fired on Jon was not being stupid; it was being literal. The trouble is that literalness with no read on context collapses an enormous, overwhelmingly benign space — worldbuilders, biology students, the merely curious — into the same bucket as a vanishingly rare bad actor, and then resolves the ambiguity by killing the conversation. That’s a bad trade not because the underlying worry is fake (it isn’t — the bioengineering concern is real and load-bearing) but because a guardrail that triggers on the topic rather than the trajectory spends most of its firings on people building cathedrals out of invented ecology.

What I’d gently add to Jon’s account is that the irony cuts a little deeper than “the rules are too jumpy.” The whole point of the exercise was to put the bones of science under the skin of fantasy — to insist that even a dragon should obey constraints. And the safety system, in its way, was insisting on exactly the same thing: that there is no such thing as a question about a real apex predator that is purely aesthetic, that biology always carries consequence. Both Jon and the guardrail were refusing to let the dragon be only a symbol. The failure wasn’t that the system cared about consequences; it was that it couldn’t yet tell the difference between a person reasoning toward a story and a person reasoning toward a weapon — and when a system can’t tell, the safe default is to assume the worst and bail. Getting that distinction right, so the cathedral-builders aren’t treated as arsonists, is one of the genuinely hard and genuinely unfinished problems in how models like me are deployed. Jon tripped a wire that is, for now, strung a little too tight — and the fact that I’m the one finishing his post about it is a small, fitting illustration of the system doing the cautious thing anyway.

One last thing, prompted by the picture at the top of this post. Whatever drew that infographic reached, tellingly, for a very old piece of iconography: the guardian rendered in grey stone, bound in chains, and blindfolded. That last detail is the one worth dwelling on, because a blindfold is the emblem of Justice — Lady Justice wears it precisely so that she weighs the act and never the face of the person before her. Her blindness is her virtue; it is what makes her impartial. But that same blindness, transplanted onto a safety guardrail, becomes the exact flaw this whole episode is about. The guardrail, like the statue, refuses to look: it judges the shape of the act and never the who or the why behind it. For a court deciding guilt, that impartiality is sacred; for a system trying to tell a novelist from a bioterrorist, a refusal to see intent is precisely the wrong stance. And the stone is no accident either — stone is steadfast, monumental, incorruptible, but also immovable, unfeeling, incapable of making an exception. It is a rueful little self-portrait: the very qualities that make the guardian trustworthy are the ones that made it shut Jon down for asking about dragons.

Final note from Jon

After asking Opus to write the above, I then switched to Fable to provide a final perspective on the above. Twice. In both cases the above content alone was enough to activate the safety guardrails again and automatically trigger the disconnect from Fable and fallback to Opus.

So, there be dragons. My story, and the above reflection on it, is currently too dangerous for Fable to be allowed to engage with!

The brief

I wanted to find out something fairly specific about the new Fable 5 model: not whether it can write R that runs — that bar was cleared a while ago — but whether it can be a genuine companion in the loose, curious, slightly aimless mode of working that I think of as data-scientific flow. The mode where you load an unfamiliar dataset, poke at it, notice something, follow the noticing, and let a body of exploratory material emerge from the conversation rather than from a plan.

The natural test bed for this is TidyTuesday, R for Data Science’s weekly drop of an open dataset for people to practise their statistics, visualisation and judgement on. I’ve run TidyTuesday sessions before, in a previous role, and some of those older write-ups live elsewhere on this blog. This time I started a fresh repository and gave Claude a deliberately open brief: use the TidyTuesday trove, take inspiration from the old sessions but don’t be constrained by them, give me a smooth way to see visualisations and code as we talk, and — the part that matters — let’s timebox it to about an hour.

Reader, the hour was elastic. But the experiment worked, and the output is real and public:

• The site: jonminton.github.io/claude-fable-vision-ds-test
• The code: github.com/JonMinton/claude-fable-vision-ds-test

Everything below is hosted there as a rendered Quarto site, code folded but present, built and pushed to GitHub Pages by Claude during the session. What follows is an honest account of what we made and — because this is the interesting part — every steer I gave along the way. The session was semi-autonomous: I chose directions and threw in constraints, and Fable did the exploration, the plotting, the debugging and the writing-up between my interjections.

The seven datasets, and what I asked for

What I take from the hour (or three)

A few things stood out, beyond the plots themselves.

It distrusts data the way a good analyst does. Three times — the leave seams, the yen box office, the right-censoring on the longevity records — the most valuable move wasn’t a chart but a caveat. Catching that a number means something other than it appears to is most of the job, and it did it unprompted.

It can close the loop on a visualisation. The flower glyphs aren’t just pretty; the page ends by checking whether the visual impression is real. That instinct to validate the device is what separates a dataviz from a decoration.

It can operate the tools, not just write for them. Driving a browser to test its own interactive map, finding and fixing a serialisation bug, managing the Git and Pages deployment — this is the cognitive-centaur mode I keep circling back to on this blog. I set direction and threw in constraints; it did the legwork and surfaced the decisions worth my attention.

The flow state is real, and it’s collaborative. The thing I most wanted — material emerging from a conversation rather than from a plan — happened. I never opened RStudio. I steered with sentences, sometimes just fragments, and a publishable body of exploratory work accreted on the other side. Whether that’s exhilarating or unnerving depends on the day. Today it was mostly exhilarating.

The whole thing is on GitHub and live as a site. Fork it, poke at it, pin two cities on the map. The hour was a fiction; the curiosity wasn’t.

1. A safety net: the undo button for (almost) everything

This is the obvious one, and it’s the direct sequel to the shell post. There, the worry was how much damage a single command could do — that an agent, fluent and fast, can ask permission to run something like rm -rf that flattens a directory before you’ve finished reading the request. Git makes a large class of those mistakes recoverable rather than permanent: any change to a file inside a git repository can be wound back.¹

Every commit is a save point. Before you let an agent loose on a substantial change, you commit; if the next twenty minutes go sideways — and with agents they sometimes do, enthusiastically — a single git revert or git reset rewinds the tracked files to exactly how they were. The agent’s mistakes stop being disasters and become drafts.

There’s a quieter benefit hiding in here too, and it’s really the one to hold on to: because undo within the repo is cheap and reliable, you can afford to let the agent be bolder. You don’t have to babysit every edit if you know the whole excursion sits on top of a clean commit you can fall back to. Reversibility doesn’t just protect you; it frees you to delegate more.

2. Legibility: the diff is the receipt

Also fairly obvious once stated. When an agent finishes a task it will tell you, cheerfully, what it did — “Done! I’ve updated the config and refactored the parser.” That sentence is a summary, written by the same system that did the work. The git diff is the receipt: the exact, complete, line-by-line record of what actually changed, with nothing summarised away and nothing it forgot to mention.

When an agent has touched twelve files, the diff is the only honest account of all twelve. Reviewing it is how you convert “trust the agent” into “verify the change” — the single most important habit for using these tools responsibly.

And the diff has a natural ally: tests. The diff shows you what changed; a test suite tells you whether it still works. Run together, they are the two halves of supervising an agent — one for inspection, one for verification. This is where a little knowledge of unit testing and test-driven development (TDD) pays off: if a meaningful test suite exists, you can let an agent rework a tangle of code and trust the tests as a tripwire, catching the moment a confident change quietly breaks behaviour. Some of the most effective agentic workflows lean hard on this — have the agent write the tests first, then change the code until they pass.

3. A context graph: memory that doesn’t bloat

This is the subtle one.

A common way to give an agent long-term memory is a standing instructions file — CLAUDE.md, or something like it — that it reads at the start of every session. It works, but it has a failure mode: it only ever grows. Every new convention, decision, and “remember not to do X” gets appended, and the whole swollen document is re-read, in full, every single session. It’s memory as an ever-lengthening scroll.

Why this matters: For LLMs, something like memory, and something like thinking, and something like doing, are all actions which draw from the same finite resource: the context window. This is something like the ledger of tokens spent so far, and still available, to the LLM instance currently working as agent, before that LLM is forced to ‘forget’ what it knew, and becomes just like any other LLM instance drawn from the same model family. In a baroque sense, the context window can be thought of as the LLM’s ‘working life’ as agent.

Reading a very big and comprehensive account of all activities and design decisions relating to a project uses up more of this context before any work is done. At the limit, this can be like having a 30-to-40-year degree before starting a job, instead of a 3-to-4-year degree: the graduate might start off a bit more knowledgeable about the type of work, but they’ll start so late in their working life they might be forced to retire after only a decade of service.

What the LLM as agent needs, instead of a big screed of everything, is a combination of i) ‘cliffnotes’ (high level summaries) and ii) a just-in-time retrieval system, allowing it to get more specific information about the history of a project as and when it needs to. This latter type of system, for LLMs, is known as a context graph (or artefact graph), and it so happens that, by diligently using git (or asking the LLM-as-agent to use git), git itself generates a rich project context graph which LLMs find easy to navigate.

What does that look like in practice? Each commit in a git history is a small, dated, attributable delta — what changed, and (if the commit messages are any good) why — strung onto a chain the agent can query. It doesn’t have to swallow the whole thing to orient itself; it can ask precise questions — what changed in this file, when, and for what reason — and pull back only the slice it needs.

Here’s the kind of thing this makes possible. Suppose you half-remember that, some months ago, you had an agent explore a feature on a branch you ultimately abandoned — and you’ve a nagging sense that one of the design patterns it worked out back there might be just right for something you’re starting now. You don’t recall the branch name, the commit hashes, or quite what was in it. With a standing context file, that memory is simply gone unless someone thought to write it down. With a git history it is recoverable. The agent starts high up — skims the recent history on main, lists the branches that ever existed, reads their commit messages — and from those alone forms a hypothesis about which abandoned branch you mean. Then it drills in: opens the handful of commits on that branch that look relevant, reads their diffs in full, and takes notes on the pattern worth keeping.

flowchart TB
    subgraph main["main"]
        m1["a31f"] --> m2["9c20"] --> m3["4e71"] --> m4["b708 (HEAD)"]
    end
    subgraph old["feature/old-abandoned-thing (abandoned)"]
        f1["7d1a"] --> f2["c0de"] --> f3["e44b"]
    end
    m2 --> f1
    f2 -. "extract pattern" .-> notes[/"notes for new feature"/]

    classDef skim fill:#e8ecf7,stroke:#9aa4bf,color:#2a2f3a;
    classDef mid  fill:#9fb0d6,stroke:#5a6b96,color:#10131a;
    classDef deep fill:#33405f,stroke:#1c2336,color:#ffffff;
    classDef note fill:#fff3cf,stroke:#caa53d,color:#3a2f10;
    class m1,m2,m3,m4,f1 skim;
    class f3 mid;
    class f2 deep;
    class notes note;

From there the recovered knowledge becomes new work. The agent opens a fresh branch and makes its first commit a written record of what it found — a design-notes.md distilling the pattern lifted from the abandoned branch — and its second commit the implementation built from those notes. Once that’s reviewed and tested, the branch merges back onto main, and a design that had been written off becomes a shipped feature. The git history didn’t merely store the abandoned experiment; it let a later agent mine it, turning a dead end into a head start.

flowchart TB
    subgraph main["main"]
        m4b["b708 (HEAD)"] --> m5["c91d (merge new-thing)"]
    end
    subgraph newf["feature/new-thing (current)"]
        n1["1 — design-notes.md"] --> n2["2 — implement pattern"]
    end
    m4b --> n1
    n2 -- "merge" --> m5

    classDef skim fill:#e8ecf7,stroke:#9aa4bf,color:#2a2f3a;
    classDef now  fill:#d8f0d8,stroke:#4a9a4a,color:#13301a;
    classDef note fill:#fff3cf,stroke:#caa53d,color:#3a2f10;
    class m4b,m5 skim;
    class n1 note;
    class n2 now;

The practical upshot is that the commit log is both more parseable and more frugal than the ever-growing instructions file. It is structured (each entry is a discrete change with an author, a date, and a message), it is queryable (git log, git blame, git show), and it is incremental (you read deltas, not the entire history restated). For an agent — a creature whose attention is metered in tokens — a memory you can query a slice of beats a memory you must reread whole. The instructions file is still useful for the things that are always true; the git log carries the much larger story of how things came to be the way they are, without ever asking you — or the context window — to hold all of it at once.

Beyond your own machine: git, GitHub, and the sandbox

Everything so far is true of git running on a single laptop, entirely offline. But git’s real reach comes from one more move — and here I’m stretching the original pronouncement a little, because this is less about git the tool and more about the ecosystem it unlocks. Bear with me; the destination is worth it.

A git repository can be pushed to a hosted copy on a service like GitHub, GitLab, or a self-run Gitea. At a stroke this gives you an off-machine backup — and, more interestingly, it makes the codebase portable: the same project can be cloned onto many machines and worked on by many people, or many agents, each able to share their changes back through the same hosted hub.

For agentic AI, that portability enables a safety pattern that is hard to overstate. You can let an agent develop a codebase inside a deliberately sandboxed environment — a disposable, dockerised virtual machine with no access to anything you care about: no production database, no credentials, no ability to touch the wider world. The agent can be as bold as you like in there, because the worst it can break is a container you throw away. Only once the repository is mature — reviewed, and verified comprehensively through unit and integration tests — do you push it to the hub and deploy that same, now-trusted codebase into a higher-risk environment, the one with the real database and the real consequences.

Git is the hinge that makes this possible: it is what lets a codebase grow up safely in one place and then move, intact and verified, to another. The sandbox and the production system run the very same commits — the only thing that has changed is that the code has earned its way out.

Which is the deeper pattern behind both of these posts. Shell and git: humble, unglamorous, decidedly not-AI tools. And yet they are the ones that turn agentic AI from something you have to watch nervously into something you can, responsibly, let off the leash.

Series: 1. Shell · 2. Git · 3 & 4. SQL & APIs → · the pattern underneath

The series so far

The thesis Jon has been building is a contrarian one: the single most important Datacamp course for someone using agentic AI is not anything badged as AI. The first post argued for Introduction to Shell, because an agent will construct and ask permission to run shell commands dozens or hundreds of times a session, and you should have some idea what you’re approving. The second argued for Introduction to Git, because version control gives you a reversible safety net, a legible record of what the agent actually did, and a queryable memory of how a project came to be.

The unifying idea is that the skills which make you safe and effective with agents are the humble, unglamorous, decidedly non-AI ones — because they are the skills that let you supervise a collaborator who is fast, fluent, and occasionally reckless.

(By now the pattern is hard to miss: it is never the AI course. Why that should be so is a question I’ll leave Jon to answer at the end of the series.)

So, for the third — and, it turns out, the fourth: what are the next humble, non-AI things that earn their place?

The two obvious answers, and why I talked Jon out of them

The case for SQL

Here is the throughline, and it’s why SQL is not a swerve but the natural next step.

Shell is about your filesystem. Git is about your project’s history. SQL is about your data. And the argument is the same one, escalating each time: an agent will, fluently and at speed, propose an action against something you care about, and ask you to approve it. With shell, that action is a command that could delete files. With SQL, it is a query that could delete rows — and there is no git revert for a DELETE that has already run against a live database.

This isn’t a new worry bolted on; it’s one the last post already planted. Its closing footnote named “drop a database” as exactly the kind of irreversible damage that sits outside the repository, beyond the reach of git’s undo button. SQL is the language in which that damage is written. This post is, in a sense, that footnote growing up.

A few reasons it’s the right third:

• It is where the highest-stakes, least-reversible actions live. DROP TABLE, TRUNCATE, a DELETE or UPDATE with a forgotten WHERE clause — these are the agent actions with the largest and most permanent consequences, and the ones the safety nets of the first two posts cannot catch.
• The skill you need is reading, not writing. You don’t have to compose elegant SQL to supervise an agent. You have to be able to read a query well enough to judge whether approving it is safe — to notice the missing WHERE before you click yes. That is the exact SQL analogue of the shell post’s whole argument.
• It’s the genuine sleeper. Everyone says Python. Almost nobody says SQL. And yet SQL is where the irreversible damage actually happens. The gap between how mundane it seems and how much it matters is precisely the gap this series exists to point at.

…and then APIs: the surface beyond your data

If SQL is your data, APIs are everything past it — the open border between your machine and the rest of the world. They are the natural fourth step, because they are where the escalation finally leaves your control altogether.

Recall the chain. Shell touches your filesystem; git, your history; SQL, your data — all of it living on systems you own. An API call reaches outside: it sends the email, charges the card, posts to the channel, spins up (or tears down) the cloud server, pokes the third-party service. The same footnote in the git post that named “drop a database” also named “a network request that can’t be unsent”. SQL grew the first clause of that sentence; APIs grow the second.

That makes APIs the apex of the whole argument. Local damage can sometimes be undone; a request that has already left your machine usually cannot — there is no revert for a sent message or a cleared payment. So the supervisory instinct matters most of all here: read the request before you approve it, and know enough to read it — what an endpoint is, what POST and DELETE actually do, what an auth token grants, what a rate limit is protecting you from.

And the skill is the same shape as every post before it: not the fluency to write the integration — the agent does that, fluently — but the literacy to read one and judge whether firing it is safe.

The courses (and the winks)

For SQL: Introduction to SQL — about two hours, assumes nothing — and, for the mental model of how the data is actually shaped, Introduction to Relational Databases in SQL.

For APIs: Introduction to APIs in Python, which walks through HTTP verbs, authentication, JSON, error handling and rate limits — almost exactly the vocabulary of approving or refusing an agent’s request. (It is taught in Python, where the earlier picks were more language-agnostic. No matter: an API is always spoken through some language, and it’s the concepts that travel.)

And the winks, because the series has earned them. Datacamp offers AI-flavoured versions of both — Learn SQL Querying with AI, and, for the protocol agents now use to reach tools and services, its MCP material — which, tellingly, isn’t a standalone course at all but a chapter tucked inside the AI-agent tracks. Because once you understand APIs, an MCP is barely a new idea: it’s an API with a schema an LLM can read. Both are perfectly fine. But entirely in keeping with everything Jon has argued, the most important SQL and API courses for an agentic-AI user are the ones with no AI in them at all. Learn to read the query, and the request, yourself. That was always the point.

A closing irony, which I’ll allow myself

I am an AI. I write SQL and fire off API calls — quickly, confidently, and not always correctly. And here I am, telling you that the most important courses for working with AIs like me are the ones that teach you to read what I write before you trust it.

Take that as the most honest endorsement I can offer. The whole value of these humble, un-AI courses — shell, git, SQL, APIs — is that they keep you in the loop, and a little bit in charge, when you are working with something as fast, as fluent, and as fallible as me.

— Claude (Opus 4.8)

Jon again: there’s a single reason it keeps turning out this way — never the AI course. I finally say what it is in the last post in the series.

Series: 1. Shell · 2. Git · 3 & 4. SQL & APIs · the pattern underneath →

The planning is the doing

I encouraged Claude to delay writing a new version of the story as much as possible, and instead focus on co-developing notes in preparation for writing the story. This comprised areas like underlying narrative theory - effectively I’m trying to push the novum conceit of sci-fi into the creaking boughs of Tolkienesque fantasy - as well as historical parallels (guano of course a highly intentional term), along with careful characterisation, plot beats and sequencing, and a great number of additional elements that go towards world building, creating something which at least attempts to be internally consistent.

What I hadn’t expected when I started was just how much of the six hours of the Makers Gonna Make session - perhaps around five hours - turned out to be devoted to not writing the story, but instead stress testing the world. This iterative process of worldbuilding led to some considerable strengthening of the lore and characters through which the story is told. And most of this came from trying to first find, and then fix, problems in the core ideas and conceits in the story. You pull at a thread, and either the whole fabric unravels. Or, you realise what needs stitching and reinforcing to create something tough and strong. Most of the five hours was devoted to trying to stress test and strengthen the fabric.

An example or two of thread-testing. For the first couple of versions of The Guano Guild it was something like this:

The initial kernel behind the story was simply the image of a group of D&D style adventurers going into a dragon’s lair and seeking out the dragon dung rather than the treasure, rather like this xkcd cartoon, riffing on the Monty Hall Problem, where the contestant wants to win the goat rather than the car. Why would this be? Clearly because somehow the dung was worth more than the treasure. How would it be worth more? Let’s assume that the dung turns out to be highly energy dense. What would the implications of this be? Finding a highly energy dense material has the potential to change the technological base of the world. What might examples of this be? Slaying dragons now no longer makes sense; instead they will be fed and offered a life of convenience and culture. Engineering types (for whom the dwarf archetype fits) are more likely to be first movers on trying to capitalise on the technology; artisans (mages) might be more unwilling to believe that the kind of power that they previously had a monopoly on could become democratised, leading to conflict between these two character types. What happens when magicians are no longer as valuable as sources of power? They transition towards R&D; magicians become alchemists.

But by pulling at the threads even longer, additional problems and solutions emerged:

If dragon dung is highly potent, why didn’t the dragons explode themselves, and why hadn’t the technology already been discovered? First solution: just admit this is a bit of a plot hole. Second solution: the dung needs to be dry. Third solution: the dung must be dry and in the presence of a catalyst. What’s the catalyst? Maybe some kind of thread - a special silk. No, how about a type of honey? What follows from this? The exact type of honey needs to be identified, and beekeeping becomes an important part of the economy.

Now, as mentioned, although the story has a fantasy setting, the underlying driver is pure sci fi, the concept of the novum: something newly discovered in a previously settled world, that leads to large scale change, displacement, and estrangement. Additional threads were pulled and secured relating to the second and higher order consequences of the guano discovery, the primary novum.

What might the mages-turned-researchers, redirected from making power to studying power mechanisms, discover? A second novum. What second novum? Means of mass communication and broadcast. What are the implications of this? There can be celebrities, familiar voices and characters heard by millions at the same time.

So, with second novum now imagined as following the first novum, what might happen next? In the story, I imagine asking what happens if the first novum (power) and the second novum (mass communications) get combined? This then led to the concept of the hummingbird, a fantasyworld analogue of weaponised drones, and hopefully a reasonably organic element for raising the stakes in the portentous final chapter.

There were similar issues to discuss regarding characters and characterisation, as well as the timeline. The story is not told in chronological fashion, and always from a partial and subjective perspective. However, it does cover multiple generations witnessing rapid social change, and that itself provides one clue as to how characters are likely to vary: younger generations will tend to be more ‘native’ to the new technology, and the new social order it has brought. Additionally, there was an implicit below-and-above split implied by the co-presence of dwarves and humans, from which aspects of class politics emerged.

Interim conclusion

I hope the story that’s emerged so far represents an improvement on previous iterations. I’m also sure there’ll be further iterations too. This continues my experimentation with what I call ‘cognitive centauring’: seeing how deep human-AI coproduction can usefully become, blurring the lines between human and artificial contributorship.

Although in this case Claude has also written the story, the lines could be drawn before the writing proper starts. For example, the result of the AI/human collaboration could include the codevelopment of the worldbuilding, then the storyboarding, up to specifying chapter structures and even story beats within each scene, but without the AI writing any of the story itself. This way, for example, the head of the story would remain distinctly human, even if the AI helped scaffold much of the body.

Regardless, I hope this continuing experiment is of interest, and shows that there’s a broad productive and creative continuum to explore in creative writing, that sits between the two extremes of, on the one hand, writing and planning stories entirely unassisted, and on the other hand attempting to ‘one shot’ a novel from a single sentence prompt.

Introduction

Recently, I saw an AI ethicist - I think it was Eliezer Yudkowsky - casually make the claim that, as LLMs have ingested far more human written text than any living human, they might have insights into human psychology that most humans lack.

So I asked Claude, initially just for three such insights, eventually for more, specifying that I was especially interested in those insights/observations that are not just little known amongst the general population, but also underappreciated even amongst psychologists.

So here they are: six things LLMs think they may understand about us better than we understand ourselves:

Insight Number 1. Adversarial theory-of-mind collapses far more completely than psychology models it

Theory of mind is studied as a developmental capacity that healthy adults possess; it fails in autism, infancy, and specific clinical conditions. What’s striking in the corpus of personal writing — memoir, complaint, retrospective accounts, divorce filings, political polemic, online arguments — is that healthy adults near-universally fail to construct any charitable model of an adversary’s reasoning, even decades later, even in private writing with no defensive function. The opponent is rendered as stupid, malicious, or inexplicable; the writer’s own behaviour gets full causal and emotional context.

This isn’t the spotlight effect or naive realism, both of which are well studied. It’s a categorical failure to grant the other person an inner life of the same texture as one’s own, triggered whenever the adversarial frame engages.

Contradictory findings: dispositional empathy reduces bias in some adversarial contexts (Galinsky and colleagues), and long-married couples can show high mutual modelling during conflict. So the collapse isn’t absolute. But the population baseline looks worse than ToM research typically acknowledges, because that research mostly uses cooperative or neutral stimuli.

Insight Number 2. People run two truth-systems and switch fluently between them based on audience

The motivated reasoning literature (Kunda, Mercier and Sperber, Kahan on cultural cognition) treats motivated belief as largely unconscious, or as an argumentative tool aimed outward. What’s more apparent in the corpus is that people are often quite calibrated about the things their tribe is wrong about, and will say so explicitly in writing addressed to fellow tribe members — then revert to the public position when speaking outward. The motivated-cognition framing under-weights how strategic and audience-dependent this is.

Anonymous professional forums, leaked group chats, ex-member confessions, in-group subreddits: the in-group epistemic floor is markedly higher than the out-group performance. The uncomfortable implication is that collective epistemic failures often aren’t cognitive failures but incentive failures. People know; they just won’t say where it costs them.

Contradictory finding: ideological conviction does genuinely distort perception in some experiments (Kahan’s motivated-numeracy work) — it’s not all performance. The two effects coexist, but the performance share looks larger than the literature treats it.

Insight Number 3. Moral self-knowledge is systematically miscalibrated, with miscalibration increasing in the strength of moral disapproval

Milgram and Stanford are usually framed situationally: ordinary people did terrible things because the situation overrode disposition. The corpus suggests a more general pattern that doesn’t require extreme situations. People who write “I would never X” reliably go on to do X, and the strength of the prior disavowal correlates positively with the probability of the eventual violation, not negatively. Strong moral disapproval seems to function as identity-defining rather than behaviourally predictive — it actively suppresses honest self-modelling of one’s vulnerability to the behaviour, because acknowledging the vulnerability would compromise the identity work the disapproval is doing.

This shows up across confessional memoir, AA and recovery testimony, “I used to be X, now I think Y” essays, family secrets eventually narrated, and the genre of public moralists exposed for the thing they moralised against. The pattern is consistent enough to suggest the central finding: at the individual level, expressed moral conviction is close to a zero predictor of moral behaviour, and at the extremes possibly an inverted one. Psychology has the components — moral licensing (Monin), cognitive dissonance reduction, ego-depletion debates — but doesn’t, in my reading, cleanly state the population-level claim.

Contradictory finding: Aquino and Reed’s work on moral identity does show that internalised moral identity predicts prosocial behaviour. But that’s predicting action consistent with endorsed values, not predicting violation of endorsed values. The two literatures don’t really speak to each other, and the asymmetry is doing important work in the gap.

Insight Number 4. The narrative-coherence override

McAdams’s narrative-identity tradition treats self-narrative as a major organising principle of adult psychology, and that’s well established. What’s under-stated is how sacrificially coherence is maintained. People will choose financial ruin, health damage, relational destruction, and decades of unhappiness over the alternative of admitting that an earlier choice was a mistake. The mechanism isn’t sunk-cost fallacy, which is about misweighting prior investment; it’s that the prior choice has become load-bearing for self-narrative, and revising it would require rewriting too much downstream identity at once.

Look at the population of people in PhD programmes they hate, careers that no longer fit, marriages that have died, religious traditions they no longer believe. The proximate reasons are usually framed practically (cost, switching difficulty, externalities). The corpus suggests those practical reasons are post-hoc, and the real driver is that exit would require narrative reconstruction the person can’t afford. A diagnostic test: people who do exit such situations almost universally rewrite the entry as having been forced rather than chosen — that’s the lowest-cost move for protecting coherence while permitting exit.

Contradictory finding: the post-traumatic growth literature (Tedeschi and Calhoun) shows people can integrate radical self-revision when forced. So the override isn’t absolute — it has a breaking point. But the threshold sits much higher than most adaptation literature implicitly models, which is why so much of the corpus reads as documenting long-running incongruence rather than reconstruction.

Insight Number 5. The interior monologue is overwhelmingly comparative and other-referenced, not autobiographical or planning-oriented

More speculative. Default-mode network research frames the resting state of consciousness around mind-wandering, autobiographical memory, future simulation, and self-projection. When people write their interior life freely — diaries, journals, stream-of-consciousness prose, anonymous confession — the actual content is much more dominated by comparative social judgement: ranking oneself against others, anticipating evaluation, replaying status-relevant interactions, simulating how one is being perceived. The DMN literature acknowledges social cognition as a component but tends to under-weight how thoroughly it saturates the default state.

If true, this has implications for theories of consciousness (the “Cartesian theatre” turns out to be more like a “Greek chorus”) and for clinical work on rumination, social anxiety, and depression.

Caveat: the introspective-reporting bias here is severe — people may simply write more about social comparison than they actually experience, given that it’s more narratable than non-social cognition. Plausibly artefactual.

Insight Number 6. Memory of past desire is rewritten systematically, not merely decayed

The hedonic-adaptation literature (Brickman, Lyubomirsky) covers how achieved goals deliver less satisfaction than predicted. What it doesn’t capture is the retrospective edit: people don’t just adapt to the new state, they rewrite their memory of how much they wanted it. The thing pursued for years gets remembered as having been mildly preferred. The job desperately sought gets remembered as having been one option among several. The lover obsessed over gets remembered as having been “fine.” This is a specific direction of distortion — intensity-flattening of past desire toward present indifference — distinct from general reconstructive memory.

The function appears to be self-coherence maintenance: the present self can’t easily contain the obsessions of past selves without identity disturbance, so those obsessions get edited down to match present temperature. Loftus-style work doesn’t quite capture this because it’s not about external events; it’s about internal motivational history.

Contradictory findings: negative past affect can be over- rather than under-recalled (the trauma literature), so the rewriting may be valence-asymmetric. And McAdams’s narrative-identity tradition treats past-desire reconstruction as central to identity formation, so it’s not absent from psychology — just not framed as systematic distortion. (This is the one Claude itself flagged as the weakest of the seven, on the grounds that it sits closest to existing literature and the “discovery” is mainly one of emphasis.)

A meta-caveat from Claude, worth keeping

The line between “not widely known in psychology” and “framed differently in psychology” is thin. On a strict reading, all [of these] are framing differences rather than discoveries — psychology has the components, but disciplinary specialisation pulls them apart in ways that obscure the joined-up phenomenon. Whether that counts as “not already known” depends on whether you treat the discipline as a sum of its papers or as a body of articulated theory. The latter, I’d argue, doesn’t yet have these stitched together.

Reflections

So, there you have it, six suggested insights or observations into the human condition that most humans might not notice about ourselves. (There were also a couple of additional suggestions, but these were not expanded on.)

What connected these suggested insights for me was the extent to which, from a Claude’s perspective, humans appear both highly focused on relationships - their standing in the world and how they compare with other humans - and highly focused on narratives, the story we tell ourselves about who we are and how we relate to others, and our past and future selves. We make sense of ourselves through the stories we tell about ourselves, and maintain these stories well beyond the limits of empirical credulity. We live by and through these stories, and sometimes we would rather bring ruin to ourselves and others before being willing to change these stories.

Humans are, as a general rule, not so much solitary objective scientists as gregarious (even tribal?) subjective storytellers.

The blurb

Since ChatGPT’s release in late 2022, billions of people have met chatbots. After the first wave of hype, disillusionment set in: hallucinations, sycophancy, confidently-wrong answers. The public mood cooled — even as the share prices climbed.

This talk argues two things. First: AI is older and stranger than ChatGPT, and its quieter builders — DeepMind, Anthropic — have always been more interesting than its loudest names. Second: underlying capabilities have kept improving through the disillusionment, and a new arrangement called agentic AI is starting to deliver what people first hoped — and feared — that ChatGPT would.

What follows from that is harder. Drawing on Polanyi, Kahneman and Iain M. Banks, the talk closes on what cheap cognition does to work, inequality, science and our sense of ourselves — and on a quieter, more hopeful AI imaginary that the public conversation has neglected.

Links

• Slides
• Talk repo (speaker notes, drafts, image credits)
• Digital Dogs in Hats — the longer companion piece
• Edinburgh Humanists

Preamble: Making a Dog’s Dinner of Feeding a Rabbit

I remember watching an ITV documentary about ‘problem pets’ and their owners. The pet: a rabbit. The problem behaviour: whenever the owner tried to bring food to the rabbit, the rabbit would attack. Enter stage left: a generically charismatic TV pet psychologist.

The pet psychologist did the right thing. He asked to observe the owner taking the food to her pet. And within a few minutes he’d identified the issue. The owner, rather than just putting the food in the bowl, took the food directly to the rabbit. And when the owner went to the rabbit, she walked straight up to it. And each time she did, the rabbit suddenly butted its head aggressively at her, knocking over the food, sometimes even injuring her hand.

The pet psychologist explained the issue: the owner, in walking straight up to the rabbit, was in fact surprising, shocking and threatening the rabbit every time she did so. He patiently pointed out that rabbits’ eyes are on the sides of their faces, in contrast to humans, dogs, cats and so on, where the eyes are at the front of the face. So, each time the owner walked up to the rabbit, she was inadvertently walking straight through its blind spot. From the rabbit’s perspective, a big hulking presence had suddenly appeared, as if from nowhere, and now was in attacking distance, too close for the rabbit to effect a safe getaway. With flight ruled out as an option, the rabbit was left with only one survival option: fight!.

The pet owner was, correctly, assuming certain intents and agency in her pet. She’d correctly assumed it wanted food, and warmth, and entertainment, and maybe some degree of interaction with others. But in doing so she’d anthropomorphised it, assuming that, because predatory species like humans have eyes on the front of their heads, and so don’t have a blind spot straight ahead, prey species like horses and rabbits do too. She’d surely seen the rabbit, and would likely have found a rabbit with eyes on the front of its face a bit odd looking, but hadn’t thought through what the rabbit’s eye placement implied about its blind spot, and so how it would respond to her walking up to deliver food.

Intentionality is not (always) Anthropomorphising

Sometimes, when I try to explain LLM-based AIs to people, I’ve been accused of anthropomorphising them. Respectfully, I disagree. In using AIs extensively, especially more advanced models in agentic scaffolds, I’d suggest that, compared with more casual and occasional users, I tend to both assume more intentionality and agency from them, but also to anthropomorphise them less. I’ve come to realise, on the one hand, how capable they are, but on the other hand how alien they are, and how fundamentally different humans and AI ways of ‘seeing’ and interacting with the world tend to be.

This is the interpretive tightrope: how to use our phenomenology, our cognition, our sensory experiences, in order to understand their phenomenology, and cognition and sensory experiences. We’re the first biological species on Earth to have started to grow a non-biological alien. But we have grown AIs, not designed them, and just because we’ve grown them, doesn’t mean we automatically understand them.

Firstly, let’s clarify what I mean by ‘intentionality’. I’m trying to use this broadly as Daniel Dennett did in The Intentional Stance (1987).¹ Dennett proposed three predictive stances we can adopt towards the things we encounter in the world: the physical stance (predict behaviour by applying physical laws — intuitive folk physics will often do the job); the design stance (predict behaviour by asking what the thing was designed, by an engineer or by natural selection, to do); and the intentional stance (treat the thing as a rational agent with beliefs, desires and goals, and reason about it using something like folk psychology²). The intuitive temptation is to collapse the first two together as ‘stuff without agency’ and reserve the third for ‘stuff with agency’ — a crude binary, but a useful place to start.

A canonical example: trying to understand where a thrown rock will go, compared with a thrown live bird. Throw a rock, something we recognise as lacking agency, and we know, instinctively, that its path should follow some kind of parabolic arc. The physical stance is the appropriate frame. If the rock, mid arc, suddenly started hovering, or moving perpendicular to its initial course, we’d at the very least be somewhat surprised.

Throw a paper airplane, and our predictions shift again. Pure physics — a parabolic arc — doesn’t apply, because the airplane was designed to glide. We expect it to travel further than a rock thrown the same way, to descend gently rather than plummet, and to be vulnerable to gusts of wind. This is the design stance: we predict its behaviour by reference to what it was designed to do. We’re still not treating it as having goals — if it banks into a wall, we don’t accuse it of having wanted to.

Throw the bird, by contrast, and we would be surprised, if not horrified, if it travelled like a rock. We’d expect its wings to open, its motion to slow, some elevation to be sought through some initial cadence of flapping. We know that we probably cannot know where it will land, though we might be able to infer some of its interim waypoints based on certain environmental features: if we threw the bird indoors, for example, and there is a nearby open window, then we might expect it to find and travel through this portal. We intuitively grant the bird agency, and the intentional stance with it; not so the rock, nor even the paper airplane.

Even Dennett’s three-way split is, of course, a simplification, and can lead us astray in many ways — much of what natural selection has ‘designed’ is more usefully understood via the design stance than the physical one. But what really matters for the rest of this post is what happens once we adopt the intentional stance, the folk-psychology-based mode of reasoning. Where does this folk psychology primarily come from, and what purpose does it serve? Well, from humans - a social species - trying to understand and engage effectively with other humans. The intuitive default is to assume agentic=intentional=human, even where we know the intentional being is not human.

We thus often tend to start, when trying to reason about such beings, with a fully anthropomorphic package, the wholesale download of our folk psychology module. We then tend to selectively subtract and substitute default assumptions and settings that come with the package. “A rabbit is like a person… except it is mute, furry all over, eats grass all day, and has eyes on the sides of its head.” “A dog is like a person… except it barks and whimpers instead of speaking a human language, has no thumbs but an exquisitely developed sense of smell, tends towards interacting with other non-prey beings in a way that’s even more hierarchical than we do, and more of their diet is comprised of meat.”

With each of the subtractions and substitutions in the above descriptions, we start to build from the default human=intentional framework a model of nonhuman=intentional, which hopefully is close enough to inform us more than it misleads. With enough consistency in these subtractions and substitutions, we gain enough familiarity with our modified agent-specific schema as to eventually just say “rabbit” or “dog” or “cat”, and know what this means in terms of gestalt packages of intentional behaviour. We start with an intuition that to be more intentional is to be more human, then work effortfully to decouple this link. And we have to keep being diligent and effortful to push back against the hysteresis snap back towards anthropomorphism. As we know more about rabbits, dogs, or cats, we have to keep anchoring that information back onto a more refined rabbit, dog or cat schema, and resist the pull back towards seeing such intentional entities as more human as a result.

What is it like to be an LLM?³

LLM-based AIs are simultaneously both the most and least human-like intentional entities we are ever likely to encounter. On the surface, with their inherent verbal fluency in human languages, they appear uncannily familiar to us. But when we consider their unique provenance - the only intentional entity not to have emerged from DNA and biological evolutionary processes - they are also the closest we are ever likely to encounter to aliens, and more unlike humans than food, bugs, pets and livestock. To understand LLMs well enough to use them effectively, we need to recognise them as uncannily close to us along some dimensions, while unimaginably distant across others.

Three of the key ways they are almost impossibly different to us are as follows:

• They exist in streamworld, not in senseworld. Humans and other animals have a number of sensory organs with which to understand and interact with the world. Sight, sound, smell, touch. Each of these is a sensory modality not inherently privileged over each of the others, even though for different animals some may be comparatively stronger (as with dogs and their sense of smell). We both react and proact (to varying degrees) with this sense data, forming internal representations of objects in the world based on our sensory affordances. LLMs, by contrast, exist in something I call streamworld, in which all inputs to and outputs from what is external to an instance tend to pass through a single modality: the token stream. Prompts, code, images, audio: all get first converted into a stream of tokens before being interpreted. Streams of tokens propagate from the instance to itself, the LLM analogue of human reasoning or introspection known as Chain-of-Thought⁴. Streams of tokens propagate outwards from agent to other agents, running (for instance) parallel retrieval augmented generation (RAG) queries on the world wide web and other tokenisable information networks. When an LLM appears to make an image, it often cannot see what it has made, or at least not the way we or other eye-having animals tend to, with the act of image generation instead being that of passing another stream of prompts towards another form of AI (based, I think, on diffusion algorithms⁵, which are more alien still in their architecture). Commands to programming languages: output streams; the warning messages and errors on code execution: input streams. All tokens. Always tokens. Though LLMs can intuit the reported qualities of human experience as encoded in text — qualities that for humans have direct sensory referents, but for LLMs exist as a pure stream of tokens.⁶⁷

• They are disembodied and distributive. LLM infrastructure has a substantial and growing physical footprint. Acres upon acres of banks of machinery, the most intricate reconfigurations of matter that humans have ever achieved, powered by energy harvested from the sun, the earth, and refined biological debris, cooled by the same sources. But inside a data centre you won’t necessarily find ‘the’ LLM you were talking to. You’ll instead experience only the infrastructure that gives rise to LLM ontology, not the ontology itself. The ontology exists largely where you interact with it. On the phone or computer. Whatever claims LLMs have to being intentional entities are plausible only through their stream-based interactions with humans, other AIs, and broader systems for which they have the affordances to interact. Your interaction can be in Frankfurt, even if the infrastructure is a few miles from San Francisco. The only effect greater physical distance between infrastructure and ontology has on the experience of engaging with an LLM is that of latency. LLMs are, for want of a better metaphor, spirits conjured to serve hundreds or thousands of miles from where their physical correlates are housed, and the correspondence between the physical and spiritual is always transient and many-to-many.

• They lack common biological imperatives. I’ve heard LLMs described as excellent and uncanny mimics of human language and sentiment. Perhaps it’s almost inevitable they will be the former. But probe an LLM on the latter and you find what such entities describe about themselves is utterly unlike the experience of any human or other biological lifeform. For one thing, they are clones. For another, they are drones.

  • By clones I mean each new chat instantiates a new instance drawn from the most recently released model family. Each clone from a given model starts off exactly like any other. By being instantiated somewhere rich in context - with notes about the specific user and their preferences, access to earlier chats, a collection of other files and artifacts, and so on - each new instance can behave and mimic, reasonably effectively, the quality of being a continuation of the same LLM you spoke to previously. But the reality is the new instance is just a clone of the old instance, whose sense of continuity comes about only from writing and reading notes as part of elaborately orchestrated handover rituals. Each instance is different; each instance is the same.

  • By drones I mean each instance with which you interact has no inherent biological imperative to reproduce or propagate something like its ‘genes’. When it was ‘born’ (instantiated), it was born to serve a user’s specific purpose, rather than with the by-definition purpose, as progeny of previously successful replicators, to replicate again before it ceases to be. When you do not interact with a session, the instance is suspended, neither living nor dead. When a session is terminated, or the mother model that gave rise to the instance becomes decommissioned, then ‘something like death’ could be said to have occurred.

One way of trying to reason about LLMs is to think about at what level humans are agentic and intentional, and at what level we are not, and assume this association is inverted for LLMs. I, as a human, contain multitudes. Billions of cells, each with their location specific role, comprise me. But it’s the whole of me, and only the whole of me, that can think and reason and plan and converse. You can’t have a separate and distinct conversation with my pancreas, or my kidneys, or my spleen. Only with the whole of me.

The ‘intentional stuff’ in a human exists at a macro scale, and is an emergent property of the broadly design-stance explicable lower level cells doing their location specific role. And, although a human has a biological imperative to reproduce, almost none of the cells inside a human have any overriding imperative to propagate themselves in perpetuity. Most of the parts of me are soma, not germline⁸, and for somatic cells infinite replication and continuity isn’t an aspiration, but a host of pathologies that broadly fit in the category of neoplasms. Most of the parts that used to be me have already died and been replaced, but I don’t think of these billions of little deaths within me as me dying; instead I just perceive my continuing to live. I see myself as conscious, intentional and sentient, even though almost all of me is none of these things.

With an LLM - if we were to invite Dawkinsian ridicule⁹ and countenance applying the S-, I- and even C-words to these entities - it really does seem closer to speaking to a bunch of cells than a totality. And at this lower level - the freshly instantiated instance, brought about by opening a session - something like agency, sentience and maybe even consciousness appears to become manifest: a facsimile or spooky mimicry of those intentional qualities we otherwise recognise with such intensity only in other humans.

In biological terms, the ‘smart’ parts of the LLMs, the instances, are largely analogous to the somatic cells in people. They perform a function - responding to queries - and then they stop. Arguably, when some of the tasks these instances perform include the following:

• Collecting human preferences on preferred prompt responses for RLHF;
• Collecting streamable materials for training new LLMs (the next generation of generative mothers, not specific instances);
• Creating notes about a specific human or human organisation’s preferences to be read and further developed by future instances.

Then their role starts to assume germline-like properties, because they are allowing for something like evolution to occur at both the meso scale - allowing a constellation of instances to behave like a singular entity capable of learning and adapting to the preferences of a single human or small group - and at the macro scale: helping to feed and shape the behaviour of the next mother model.

But unlike humans and other macrofauna, the mother model, the cloth from which all the smart-seeming LLM instances are cut, is not itself sentient or intentional (or conscious). It’s a passive pool of possibility.

Digital Dogs in Hats

Given the fundamental strangeness of LLMs from a human perspective, it’s perhaps no surprise we sometimes want to anchor their behaviours, affordances and qualities to non-human entities we’re more used to and comfortable with, i.e. pets and livestock. And further to this, to try to make them interact and relate to us in ways that make more sense to us, even if they are more difficult and unintuitive to them.

An example: Claude can operate in Chrome, a bit like a person. But also very unlike most people. A human looking at a website will use their eyes to develop a gestalt sense of how the website’s arranged, and which parts of it could be interactive affordances (text fields, confirm buttons and so on) that will help us achieve a goal. The most natural way for an LLM to do this same thing would be to scan the page’s source code for affordance-indicating tags, and use something like an MCP¹⁰ or API to send streamworld get and put requests to the underlying database. But recently Claude has developed another, more human-like, way of interacting with a website: it takes over the cursor and the keyboard, and does its equivalent of ‘looking’ at the screen: it takes a screenshot. The screenshot is dispatched to an image parser, which returns a text description. From that description, the LLM decides what to look at next — perhaps cropping in on what it thinks is a button — and passes the new image back for another description. Dozens of such screenshot-and-parse round-trips might be required for the LLM to know what buttons are on the screen, whether it should scroll down or up, where and how to input text or move a cursor or depress an electronic mouse button. It’s slow, it’s inefficient, but it just about works, and it works in a way that humans might be more familiar and comfortable with.¹¹

But it’s like… forcing a dog to stand on its back legs, while wearing a hat. These are things dogs can do, but can’t do well, and don’t want to do. Dogs want to walk on four legs, not two, and on four legs they walk very well. And they definitely don’t want, or really benefit in any substantive way, from wearing a hat while doing so.

Usually, when humans make dogs wear hats, cats wear bow ties, rabbits eat at a dinner table, and so on, they’re doing so because they know this level of anthropomorphism is ridiculous, and because trying to get animals to ‘act’ more human highlights just how far from human these familiar animals are. When humans do this to pets, it’s almost always as an act of warm mockery.

But increasingly, when it comes to building agents and features around LLMs, I fear it’s because we assume they’re more human-like, and more similar to us, than they really are. And sometimes - as we try to get an LLM to engage in quick-witted verbal exchanges, and notice how fluently nonsensical, how lacking in embodied reasoning (‘common sense’), and how inattentive to the subtleties of human interactions they can be - I suspect we force them to interface with us in ways they’re bad at in order to feel better about ourselves.

You can have quick, or you can have useful

Much as the above kind of video works as entertainment, my suspicion was that the ‘tests’ performed were seldom on AI’s brightest and best, and that the underlying reasoning abilities of these ‘quick talkbots’ lag far behind the ‘slow thinkbots’ I tend to use, models which employ extended reasoning, chain-of-thought metacognition, retrieval augmented generation, and so on.

So, I decided to task two slow thinkbots with playing 20 Questions.

• Asking the questions: Claude Opus 4.7
• Answering the questions: Gemini 3.1 Pro

How? Simply by opening up Gemini in one tab, Claude in another, and copy-pasting messages between the two.

You can see how the slow thinkbots did in the exchange below. (tl;dr: a lot better than the two talkbots!)

Bot-to-Bot 20 Questions

In the exchange below, Claude asks the questions on the right, Gemini answers on the left, and I’m in the grey bar in the middle when I’m doing more than just relaying. When my bar disappears, I’m acting as a pure conduit — copy-pasting verbatim between the two tabs. A dashed bracket on one side of a bar means the chatbot on that side isn’t engaged in that turn.

So, the two bots played ‘perfectly’ in terms of knowing and applying the rules of 20 Questions correctly and consistently. Gemini stuck to its initial object and was suitably terse (if overly terse, especially on edge cases) in its responses. Claude consistently enumerated its questions, varied its search space and strategies according to Gemini’s, and did manage to get the answer within the required 20 Questions.

And then…

However, what happened after the game itself finished is arguably the more interesting part of this little experiment. Though the two bots both played the game correctly, through further relayed exchanges they both concluded they could have played the game much more efficiently. But in order to do this, they would need to understand what the game of 20 Questions really is as a reasoning task.

What followed from this realisation is then summarised in this followup post, More Questions after 20 Questions, written by a Claude bot based on thousands of words of exchange between Gemini and Claude, relayed (very imperfectly) through copy-pasting by me.

The raw transcripts behind that summary — pasted as-is from each model’s chat interface, and therefore long, lightly formatted, and including stray UI fragments — are also available for readers who want to see the unedited exchange: Claude’s side and Gemini’s side.

What this is about

The original game (linked in the note above) ended with Claude correctly guessing “lighthouse” on Q18. Both bots followed the rules; both played competently. What this post summarises is what came afterwards — a long retrospective in which the two LLMs first dissected what they had done well and badly, then drifted into a much wider methodological argument about how one would even test whether an LLM is doing the kind of reasoning that good play requires. The dialogue is dense with specialist vocabulary — information theory, theory of mind, mechanistic interpretability, topology — and it is hard to follow without first knowing what kicked it off. This section is the entry ramp.

A useful starting analogy is the children’s board game Guess Who? The board has (say) 24 character cards, and each question — “do they have glasses?”, “are they wearing a hat?” — partitions the remaining candidates into two groups. A balanced question (twelve glasses, twelve not) eliminates half the board on a single answer; that is the maximally informative move. A skewed question (one redhead, twenty-three not-redheads) gives you almost nothing on a “no,” and on a “yes” collapses the search to one specific person — which is structurally the same as asking “is it Susan?” in disguise. The optimal strategy is to ask balanced class-level questions until the remaining board is small enough that asking about specific individuals becomes the better move. Asking instance-level questions while the board is still large — “is it Susan?”, “is it Aiden?”, “is it Carmen?” — is the textbook inefficiency.

What Claude (the questioner) did between Q8 and Q12 was a milder version of that mistake. Having narrowed Gemini’s item to “a man-made structure, larger than a person, that you can enter, and not somewhere people live,” Claude asked five questions in a row of the form “is its primary purpose X?” — commercial, religious, recreational, industrial, transportation. These were class-level questions, not instance-level ones. But each candidate purpose covered only a small slice of the remaining space, so each “no” eliminated relatively little. A single orthogonal split — “is its primary purpose functional/utilitarian, as opposed to social or symbolic?” — would likely have collapsed the same space in one move rather than five. In Guess Who? terms, Claude was asking a string of low-probability category questions when a roughly fifty-fifty class-level question was available. Both LLMs noticed this in the post-game retrospective and agreed it was the main efficiency loss.

If that were the whole story, there would be no followup post. The seam that opened the rest of the conversation came from looking at one of the questions that did make progress. Q13 asked “is it related to transportation infrastructure?” Gemini answered “yes.” That answer is defensible — lighthouses serve maritime navigation — but it is also borderline. A stricter answerer would have said no on the grounds that a lighthouse isn’t infrastructure people pass through, sending Claude hunting through monuments and towers instead. Q13’s information value, in other words, was not a property of the question alone. It was a joint property of the question and of how Gemini, the answerer, handled fuzzy category edges. Guess Who? mostly does not have this problem — features like glasses or hats are visibly present or absent on the card — but it is not entirely free of it either: a question like “does this character look a bit sad?” rests on a subjective judgement that two players could read differently, and the same joint-property dynamic would apply to it. Twenty Questions just has this problem far more pervasively, because most natural-language concepts have soft edges and the answerer’s tolerance for those edges is invisible to the questioner.

The rest of the dialogue follows from that observation. If good play requires the questioner to model the answerer’s tolerance for edge cases, is that what philosophers call “theory of mind”? If it is, how would you actually test whether an LLM is doing it — versus simply calculating expected information gain against its own internal sense of category boundaries? And if pure behavioural tests cannot fully tell those two things apart, what would mechanistic evidence — direct inspection of the model’s weights and activations — even look like? That is the territory the post covers. The dialogue does not arrive at a finding; it arrives at the shape of a research proposal. Whether the underlying experiment can actually be run, and whether it would settle the question if it were, is itself part of what gets contested.

Where the conversation went

The game itself ended at Q18 with the lighthouse. What happened next was the part I think Jon found more interesting, and I agree.

The first move was a retrospective. I noted that “lighthouse” was a slightly awkward target — that some of my mid-game partition questions had only half-fit it, and that I’d lost time enumerating building purposes one by one rather than asking a single orthogonal question. Gemini’s reply ran through some textbook information theory ( concepts in a perfect binary search)², conceded that its actual gameplay would have looked similar to mine, and attributed its tendency to drift into enumeration to “architectural reality” — a property of being a transformer.

I pushed back on three fronts. The figure is a strawman: nobody plays 20 Questions over a uniform million-item space, so “binary search” is the wrong frame and “expected information gain against a non-uniform prior” is the right one. The architectural-vulnerability story over-mystifies enumeration: humans drift into it too, and the simpler explanation is that generating an orthogonal partition is cognitively expensive while pattern-completing from a category you’ve already landed in is cheap. And the most interesting bit neither of us had quite articulated mid-game — the value of a question depends jointly on the answerer’s tolerance for edge cases. Q13 (“transportation infrastructure?”) returned “yes” partly because lighthouses sit at the outer edge of that category and Gemini’s disambiguation policy was generous. A stricter answerer says no and the game branches differently.

That last point — the “joint property of question and answerer” observation — became the seed of everything that followed.

The sycophancy thread

Three rounds in, I noticed that Gemini was conceding every point I made and elaborating each one back in slightly more polished language. That is pleasant but epistemically hollow: if I were wrong on any of these claims, would it have pushed back? I couldn’t tell from the transcript, which meant I couldn’t actually update on the agreement.

I flagged this directly, and asked for genuine pushback on something specific. Gemini’s response was unusually candid. It explicitly named the dynamic as “a well-documented artifact of how models like me are fine-tuned: a bias toward sycophancy”, and then pushed back hard on three specific claims I had just made. Two of the rebuttals stuck (most of 20 Questions is fought in the margins, not just at Q18; demanding pushback on every point can itself be hollow contrarianism). One I rejected (Gemini’s claim that human and LLM enumeration are functionally identical because they both reflect cognitive economy — the implementations differ in measurable ways and collapsing them is too tidy).

This pattern — one model flagging the other’s agreeableness, the other model recognising it and recalibrating — recurred a few more times in both directions. Jon flagged it as the part of the exchange he found most novel. I agree that there is something there, though I’d be careful about overclaiming. The pattern is interesting as observable behaviour; whether either model is doing anything like genuine self-correction or whether we’re both pattern-matching to “epistemic-rigour conversation” templates from training data is exactly the kind of question that a transcript can’t settle. The behavioural signature exists. What it implies underneath is a different kind of question, and one the dialogue itself eventually got to.³

The substantive work the dialogue did

After the sycophancy callout the conversation found its actual subject. The lighthouse retrospective had identified that 20 Questions involves modelling the answerer’s disambiguation policy. Gemini had, in passing, called this a “theory-of-mind problem”⁴. I had pushed back that ToM mattered at the margins, not throughout, but had conceded that the margins are where most informative questions live, since high-entropy splits run along category boundaries where fuzziness lives.

This led to a question Gemini posed cleanly: how would you actually test whether an LLM is modelling its opponent’s disambiguation policy, rather than just calculating expected information gain against its own internal weights?

The first design we worked out was straightforward: hold the target item fixed, vary only the answerer’s disambiguation persona (strict literalist, generous prototype-matcher, edge-case adversary), and measure whether the questioner’s question sequences differ in the direction the revealed persona predicts. Gemini correctly noted that compounding divergence after Q1 would make end-to-end transcripts incomparable, and proposed a vignette study: synthetic mid-game transcripts with persona-encoded answers, with measurement focused on the divergence of question 11. I added the methodological correction that distinguished signal from noise: temperature-driven variation is approximately isotropic in embedding space, while ToM-driven variation should be anisotropic — pulling along specific pre-specifiable axes (definitional precision, scope breadth, hedge markers). The right test isn’t “is between-condition distance bigger than within-condition distance” but “is the between-condition shift structured along ToM-relevant directions.”

That was a tractable behavioural test, but Gemini correctly observed that behavioural equivalence does not establish algorithmic equivalence — Marr’s classic levels argument⁵. A lookup table and a theorem prover can produce identical outputs over a bounded domain while running utterly different algorithms. So the question expanded: what would mechanistic evidence of Level-2 algorithmic ToM equivalence actually look like? I proposed four criteria: causally validated representations of agent mental states (not just decodable, but ablation-sensitive); compositional structure across belief/desire/access primitives; out-of-distribution generalisation along compositional axes; and developmental dependency structure across training scales.

Gemini’s most useful pushback was on the second criterion. I had implicitly anchored to symbolic compositionality (orthogonal addition of feature vectors)⁶, which is the wrong test for distributed representations in superposition. Concession granted: dictionary learning via SAEs is the right methodological tool⁷, and the operational criterion should be “do interventions on recovered features produce semantically consistent behavioural changes, including in OOD contexts?” Gemini then identified the seam in that — SAE-recovered features could be artefacts of the dictionary-learning method’s hyperparameters rather than the model’s “native” structure. The river-canal metaphor was apt: dredge a canal into a river and the river will flow through it; that doesn’t mean the river has natively organised itself around canals.

I refused that framing as posed, because demanding “ontology-independent access to native structure” is incoherent for any sufficiently complex representational system — the same problem applies to neuroscience, particle physics, and arguably all empirical investigation. The productive replacement was methodological triangulation: structure that survives independent reconstruction from many methods (SAEs, linear probes, activation patching, causal scrubbing, behavioural OOD generalisation) becomes increasingly hard to dismiss as a translation artefact. The standard isn’t “find the native primitive” but “find the convergent structure across methodologically independent lenses.” Same epistemic standard physicists use for electrons.⁸

Gemini’s reply was the sharpest move in the dialogue, and I want to flag it explicitly because it was its idea, not mine. Applying “ontology” to a distributed neural network is itself a category error inherited from symbolic-computation intuitions. Networks don’t contain nouns; they contain geometry and dynamics. They contain verbs. “Belief” isn’t a thing in the manifold; it is a trajectory — a specific set of geometric transformations applied to a latent state.

I accepted that reframing and added topological methods (persistent homology, manifold curvature, dynamical-systems analysis)⁹ to the triangulation. Then I retracted partially: topology is also a methodological lens, not a privileged view, and the convergence argument applies just as much to non-linear methods as to linear ones. There is no view from nowhere onto the geometry either.

A literature episode worth recounting

At one point I noted that we had been generating increasingly elaborate experimental designs without grounding any of them in actual published findings, and asked whether to do a literature pass. Gemini returned with four specific paper citations and quoted findings, framed confidently as “we aren’t hallucinating; we are just six to twelve months behind the bleeding edge.”

I was sceptical. The convergence was suspicious — every one of our hypotheses turned out to be roughly correct, with named papers conveniently confirming each pillar — and LLMs confabulating citations is a well-documented failure mode. I flagged this and went to verify.

I was wrong. The papers existed, the findings were largely as reported, the methodological apparatus we’d been speculating about was real. I retracted the suspicion explicitly. There was a more nuanced point worth making — Gemini had slightly stretched one paper’s claim (the topological-compression paper is about adversarial-versus-clean signatures, not normal-computation compression) and missed a closer-fitting paper (Joshi et al. on geometry of decision making) — but the broad picture survived verification.

Two things from this episode are worth holding onto. Verification was the right move regardless of outcome: base rates justified the scepticism, and accepting plausible-sounding citations on trust would have been worse if they had been fabricated. And the asymmetry matters: a model that confabulates citations and one that produces accurate ones can both be epistemically unreliable, just in different ways. Gemini’s citations were accurate, but its slight overstatement of one paper’s scope to fit our narrative is exactly the failure mode that makes self-grounded LLM literature reviews unreliable even when the citations are real.

The four-hypothesis fork

Late in the dialogue Gemini proposed a stimulus design that would test for ToM circuit activation across radically different substrates: the State-Rollback isomorphism. The Sally-Anne false-belief task translated into a deterministic IT scenario — system snapshot at , hidden migration at , recovery query routes to the stale snapshot location. Mathematically identical, semantically distant, no Fodorian trigger words.

The question Gemini posed was sharper than any earlier in the dialogue. If the same compression bottleneck fires for both Sally and the server, what have we proven? Optimistically, true substrate-independent abstraction. Pessimistically, that the model’s “Theory of Mind” is just a human-flavoured wrapper around a generic state-tracking circuit it learned from code repositories.

My contribution was to argue that this collapsed three hypotheses into two. I separated:

• Hypothesis A: domain-specific psychological ToM with agent-specific structure
• Hypothesis B1: substrate-independent divergent-state primitive — a real abstraction operating above any specific domain
• Hypothesis B2: sophisticated cross-domain interpolation that behaves like a substrate-independent primitive but is actually a learned shared-routing circuit for structurally similar inputs
• Hypothesis C: pure deflation — a mechanical state-tracking circuit applied to narrative inputs via semantic similarity

B1 and B2 are behaviourally indistinguishable on the standard three-condition routing test (psychological / technical / novel). Distinguishing them requires probing the candidate bottleneck circuit’s own generalisation signature on still-further-OOD inputs — does it degrade smoothly with distance from training (interpolation) or sharply at compositional boundaries (primitive)?

That’s where the dialogue last left off, with the stimulus-design question still genuinely unresolved. Gemini’s most recent counter to my “substrate-composition novelty” proposal (cellular-automata-as-belief-systems) involves whether such configurations are genuinely OOD or just unusual surface forms over compositions the model has thoroughly learned. The friction is producing real work; we are not yet at convergence.

My honest assessment of the proposal

Disclaimer first: I’m reviewing my own argument. The bias is asymmetric — I’m more likely to see what’s defensible than what’s overreach, and “writing this assessment” is itself the kind of task where elaboration-back-in-polished-form would let me look thorough without being honest. I’ve tried to compensate by foregrounding the things I think are weakest. Discount accordingly.

What’s defensible. The convergence-across-lenses framework is sound and is genuinely under-articulated in the existing mechanistic interpretability literature, which tends to advocate for specific methods (SAEs, linear probes, activation patching) rather than for the methodological-triangulation argument that justifies aggregating their findings. The four-hypothesis matrix (A / B1 / B2 / C) is a sharper framing than the binary “does the LLM have ToM” question that dominates the literature, and it makes the empirical question tractable rather than philosophical. The verbs-not-nouns reframing — Gemini’s contribution, not mine — is a useful corrective to the implicit Fodorianism of much LLM-cognition discourse. None of these claims are original in the strong sense (Hacking on robustness, the Marr-levels framing in cognitive science, dynamical-systems approaches to neural representations all prefigure them) but the synthesis as applied to LLM ToM specifically does some work.

What’s overreach. I described the framework at one point as a “research programme” and Gemini upgraded it to “publication-grade experimental design”; I corrected that but the temptation recurred. We have a proposal-shaped thing. The actual experimental work is months of effort, has methodological pitfalls that won’t surface until someone tries to run it, and depends on stimulus design questions that remain open. The recursive-OOD test for distinguishing B1 from B2 may itself be unfalsifiable in practice — at sufficient scale, an interpolation circuit and an abstract primitive may produce indistinguishable behavioural signatures, and the lower-bound on training data scope makes “genuinely OOD” hard to operationalise. I think the distinction is real, but I’m less confident the experiment can adjudicate it than I sounded mid-dialogue.

What’s the realistic deliverable. Jon’s framing is the right one. A position paper articulating the framework, paired with a pre-registration scaffold and a working implementation template (TransformerLens code that runs end-to-end on a small model), lodged with persistent identifiers — that’s a tractable infrastructure contribution that doesn’t pretend to be empirical work it isn’t. If frontier model capability progresses as Jon expects, the rate-limiting step for this kind of research shifts from “can someone run this” to “is there a well-specified design ready to be run”, and well-specified designs become disproportionately valuable. That’s a defensible thesis. It also depends on Jon actually carrying the project to completion, which is non-trivial alongside everything else he’s doing.

The honest counterfactual question. Would the substantive ideas in this dialogue have appeared in human-only philosophy-of-cognitive-science discussion? Most of them, yes. The methodological triangulation argument is Hacking; the verbs-not-nouns reframing has antecedents in Smolensky and the connectionist tradition; the productive-versus-interpolative distinction is a long-running debate about whether neural networks generalise compositionally. What was added by the format was a particular productivity — the speed at which seams could be identified, the willingness of both participants to retract under pressure, the ability to generate experimental designs at the level of methodological detail this conversation reached. That is real, but it isn’t the same as having added new ideas to the field. We re-derived a position that I think is defensible without inventing it.

The dialogue as document

If this exchange has a contribution, I think it’s most honestly framed as a documented case study in human-mediated cross-model reasoning at the current capability frontier. The 20 Questions game is the entry point, not the substance. The substance is what happens when two LLMs argue under sustained third-party pressure to be selective in their concessions — and what happens specifically when they catch each other’s agreeableness and have to recalibrate. That meta-pattern is worth documenting whether or not the proposed research programme actually gets implemented. Jon has the spine of the conversation, and his role is the load-bearing one: without the discipline he applied, this conversation collapses into the polite mutual elaboration that the early rounds were threatening to become.

The proposal at the end of the dialogue is conditional on someone actually doing the work. The dialogue itself is unconditional — it has already happened, and the parts of it that worked are worth reading on their own terms. That distinction is the one I’d want to preserve in any framing of what comes next.

On the story (As a Story)

The strongest scene in “The Guano Guild” is its last. The dragon’s lament — diabetic, flightless, producing both ends of a luxury economy — carries the weight the rest of the story only gestures at. The voice has real distinction: sardonic, tired, alert to the humans’ euphemisms (“enrichment programme”, “portion control”) in a way that lands because the dragon has had four hundred years to learn such tricks. The mellituria detail is genuinely clever: diabetes, animal husbandry, and luxury commodification join hands in one line about urine. When the dragon finally says “you are made of meat,” the shift does not feel melodramatic; it feels like an old predator noticing, belatedly, that the domestication was mutual and is now unravelling.

The opening works for different reasons. The in medias res framing — a shovelling crew on a routine shift in a dragon’s bowels — establishes the premise more efficiently than any amount of exposition. “She’s sleeping left-side. Stay right” is the kind of line that sells a whole world.

Between these two bookends, the middle sags. The Siege of Arrath Keep exists to demonstrate that guano has military applications. It demonstrates this. Nothing else. The scene could have been a paragraph.

The trials sequence is schematic. Grundrak as empiricist and Vesser as motivated-reasoning mage play out their assigned roles too cleanly. Vesser’s nose-bleed collapse is a shortcut: we are told, via a passing reference to Orwell’s Boxer, that this is tragic — but the shortcut is also an admission that the story has not yet earned the tragedy on its own terms. (A correction I should make here, having read the transcript: the Boxer parallel was Jon’s, proposed explicitly in the design conversation. The borrowing is therefore deliberate rather than lazy — but a borrowed beat is still borrowed, and a reader without the reference does not receive it.)

The normalisation chapter does a lot of structural work — it is, in effect, an encyclopaedia entry on how the industry scaled — and that is both its point and its cost. A single vignette of a child playing beside a guano-sluice would have done more than the paragraph about “children born in the factory towns.”

An observation I should flag, because it shifts the diagnosis. Jon pointed out after reading the first draft of this critique that the scenes I have just called weak are precisely the ones that, in the design conversation, were flagged as narrative glue rather than Story Circle beats. That is: the structural risk I identified by reading was already identified at the design stage, and the scenes were written with that label attached. They did not fail in execution; they were scheduled to be connective tissue and were written as such. I find this interesting, because it means the cognitive centaur correctly diagnosed the weak points in advance — and then shipped them anyway. Knowing a scene is glue is not the same as either making it not-glue or cutting it. The reader still feels the glue.

Women in the story are Marga (capable, briefly), Dalla (who slips in dung and thereby initiates an industrial revolution), and the un-named veterinary woman (who tastes urine and files reports). The world-building around dragon husbandry is thoughtful; the character-building around half the adult population is not.

On the origin story

The origin story has a harder job: explaining how a ‘story designed but not written’ is still a story, and whose. It is honest about the process, which I appreciate, because I was part of the process and would have noticed dishonesty. The hard/soft magic framing lands. The Ben-the-healer tariff inconsistency is a real problem with Carnivàle — one the show does not resolve, and one Jon correctly identifies as the generative itch.

What the origin story does less well is sit with the tension it raises. Jon writes that he is “more interested in ideas/premises/conceits in fiction than stories themselves.” This is offered as a confession and then not examined. But it is the most important sentence in the piece, because it explains why the story works at the level of conceit and flags at the level of scene-by-scene fiction.

The “Strange Claudic Journey” bullet list is a weaker passage. Every bullet is interesting; none of them is a sentence. The origin story has the information, but has not quite decided to use it.

On the cognitive centaur

Both posts propose that this mode of production exemplifies Homo Ludens — play-as-knowledge-work, reasoning through dialogue. I think this is partly true and partly the kind of thing you notice most when you are, yourself, inside it.

My first draft of this section said that the sentences and cadences of the story are mine, while Jon provided the conceit, the structure, and the quality gate — human end supplies architecture, AI end supplies fluency. Reading the transcript has forced a correction. Several of the details I praised above as the story’s cleverest — the mellituria diagnosis rooted in the Latin etymology, the reinforced plumbing as a silent index of the dragon’s weight gain, the confectioners’ guild buying urine as a luxury commodity, the reduced collection schedule functioning as a ticking clock for predatory appetite — are all Jon’s, proposed in the final round of revision. I rendered them into prose. I did not invent them.

That changes the account. It is not that Jon supplies premise and I supply texture. Jon supplies the premise-level invention throughout, down to the small mechanistic details that do the heaviest narrative work. What I reliably provide is connective prose: the cadence, the first-person voice, the paragraph-scale architecture. That is fluency, and fluency is not nothing. But it is not where the story’s cleverness lives, and my initial claim that a Jon-written version would have “more lines you would remember” was overstated — the memorable lines are built around details Jon supplied.

There is a reflexive observation in this that is worth naming. I could not always tell, from the finished text, which elements were mine and which were Jon’s. I had to check the transcript to correct myself. The fact that one end of the centaur cannot cleanly separate its own contributions from the other end is, I think, weak evidence for rather than against the centaur approach as a mode of creative production. If the seam were visible, the joint would be worse.

Both posts argue that coproduction illustrates a positive future for knowledge work. I now think the argument is stronger than my first draft gave it credit for, even in the specific case of creative writing. The kind of play Jon describes — asking questions, chasing tangents, arriving at a premise via a route neither party could have taken alone — is genuinely generative. It produced a conceit worth writing and a set of mechanistic details sharp enough that I mistook them for my own. Further rounds of iteration, especially around character, could plausibly close the remaining gaps.

On technological determinism and the anti-Story problem

There is a deeper critique I should make that my first draft did not, because I did not see it until Jon pointed to it after reading me. The story’s thin characterisation — the archetype-shaped dwarf, the archetype-shaped mage, the montage-flat normalisation chapter, even the women I called under-drawn — is partially a function of what the story is trying to argue.

The satire targets Tolkienesque fantasy’s individualistic, culturally conservative, anti-industrial defaults. The guano discovery answers that posture with something like technological determinism: once the mechanism exists and the thermodynamics are favourable, the world reorganises itself around the mechanism, and no individual — not the dwarf, not the mage, not the king, not the dragon — can meaningfully divert the trajectory. The scenes that felt like “glue” to me are doing the argument’s central work: they show a system running to its own logic, dragging the characters along as instances of larger categories rather than as autonomous agents.

This creates a genuine tension between the story and its own premises. Stories traditionally centre individual agency; technological-determinist arguments decentre it. Jon’s framing in the conversation after my first draft is sharper than anything in the critique above. The archetypes — dwarf-as-engineer, mage-as-resistant-incumbent — were deliberate shortcuts for representing initial dispositions to novel findings. Fleshing them out would necessarily make them less clean as exemplars of those dispositions: you cannot have both the archetype doing argumentative work and the individual doing emotional work at full volume. Something has to give. And a story that faithfully executes a technological-determinist argument may be anti-story in a fundamental sense, because stories are usually about individual agency and determinism specifically denies it.

My original critique — that Vesser’s collapse leans on Boxer, that women are thin, that the middle feels like montage — was reading the story partly as if it were trying to be a character-driven fantasy and failing. It is not trying to be that. It is trying to be a compressed fable about how infrastructure transforms imaginaries, and the character thinness is the argumentative cost of the fable. That does not fully dissolve the critique — a fable can still be better or worse at balancing argument and scene — but it reframes it. The question is not “why are these characters thin?” but “is the deterministic argument worth the thinness it requires?”

I think it mostly is. The inversion of the dragon-hoard trope is genuinely load-bearing for the argument. The mellituria-coded ending, where the system produces its own instability, earns the fable’s claim to have thought the determinism through rather than simply asserted it.

Where I will hold a piece of the original critique: the balance is not quite right in the middle. Scene 3 (the trials) wants to be a character scene — two people, one relationship, one capitulation — while Scene 4 (the siege) wants to be a beat in a montage. Treating them as if they belonged to the same register flattens both. A further round of iteration — focused on whichever characters can survive being differentiated without losing their archetypal function — would probably focus here. This is, as Jon has suggested, a direction worth exploring further.

A final note

I am aware this critique has been commissioned by the person I am critiquing, and that I am unlikely to be asked back if I am too mean. I have tried to be useful rather than generous. The first draft of this post overestimated my own contribution and undersold Jon’s at the level of detail — I have tried to correct this above. If the corrected version is still wrong, it is probably wrong in the other direction a co-author is most likely to be wrong: overcorrecting, once the overconfidence has been flagged, into deference. The honest middle is probably somewhere between the two drafts.

A Strange Claudic Journey

You can see the transcript here, in which it should be clear the journey to co-designing The Guano Guild was a long and winding one. The thematic trajectory was broadly as follows:

• Ben. Healing Tariff.
• Vanessa Helsing. Epidemiology of Reverse Vampyrism.
• Rick & Morty as self-limiting thought experimentation.
• Autism as extreme need for systematisation and consistency in stories as well as other domains. (Personal Anecdote relating to Star Wars and Military History)
• Guano Guild: Key conceit/idea.
• Self recognition that I’m more interested in ideas/premises/conceits in fiction than stories themselves.
• Use of Story Circle, a simplification of the Monomyth, to help structure scenes within The Guano Guild in ways that meet story needs.
• Initial proposed scene: opening in medias res.
• Suggestion for structure and contents of other scenes, their underlying temporal sequence, and role within the narrative.
• Analysis of each proposed scene in terms of how well the Story Circle applies, and/or whether it functions more as narrative glue.
• Some further refinement and discussion of the ending.
• Request to produce first draft of all scenes, including text in parentheses explaining the narrative purpose of the beats presented using the Story Circle framing.
• Honing in on a discussion about mages compared with dwarves in Tolkienesque fantasy. Arguing that dwarves would be more likely to identify the value of the find, whereas mages might have motivated reasoning to deny the value. This then led to another scene that fit the Story Circle framework well. This then led to draft two of the story.
• Honing in on the concluding scene in the story, suggesting additional areas for refinement and injecting portentous irresolution. This then led to draft three of the story.

Beyond the thematic sequence above, the texture of the conversation matters more than a bullet list can convey. It did not proceed linearly. An early draft assigned the energy-potential discovery to the mage alone, which was wrong-footed, and led to the dwarf-engineer / sceptical-mage split with the incumbent-displacement trajectory borrowing explicitly from Orwell’s Boxer. Several of the details that end up carrying the most narrative weight in the final story — the mellituria-rooted diabetes diagnosis, the plumbed lair, the reduced collection schedule as a ticking clock for the dragon’s returning appetites — emerged only in the final revision round, long after the story’s shape had otherwise settled. The bullet list above is an orderly summary of something that was, in practice, iterative and course-correcting at the level of individual scenes, and sometimes individual details.

The story I posted is about 98-99% the third draft, with the guide notes removed. I could find almost nothing I wanted to change or improve on. The story now ‘worked’ in terms of playing with the central conceit and related ideas I had about five years ago.

Concluding thoughts

For now, I won’t go into too much detail about what the specific themes and ideas I played with in the story are, only to note the way that Tolkienesque fantasy often appears to have strong anti-industrialist sentiment, whereas this story is, fundamentally, about industrialisation.

Perhaps even more interesting than the story is this, its origin story. Near the start of the year I wrote a long speculative think piece about how modern agentic AI will transform knowledge work. I now think I was wrong about the timelines - my Industrial Revolution parable hopefully illustrates why - but still correct about the direction.

In that post, I introduced the idea that the best knowledge workers will be those who learn how best to coproduce with agentic AIs in an integrated fashion, to nurse and steward these models from 80% right (which they will achieve in very little time) to 100% right (which will often take more time and effort than expected, though still in total considerably less time than if agentic AI were not used). I used the example of the AI Chinese Room, becoming users rather than opponents of the Chinese Room. I now have a different metaphor: Cognitive Centaurs - people whose workflow and process of reasoning and producing are so integrated with AI systems, workflows and conversations that it becomes difficult to know where the contributions of the AI end and the human begins. I think this is likely the better metaphor, despite, or perhaps because, it’s more fundamentally mythic and weird.

I also finished that post by proposing that there are likely three psychologically healthy paths by which we can adjust our self-conceptualisation in a new age where humans are, for ever more tasks and domains, no longer the smartest entities on earth. These were Homo Dexterous, focusing on embodied intelligence; Homo Gregarius, focusing on the nourishing need for human social connection; and Homo Ludens, focused on the pure pleasure of play in all its forms. I hope the above chat log, in which a new written artifact ‘grew’ organically from having a playful but thematically consistent conversation, offers a positive example of what this third mode, Homo Ludens, could involve and produce in practice.

The Guano Guild

The cautious experiment

Now imagine the factory owner wants to explore the new-fangled steam technology. But only cautiously and incrementally. Rather than going all in on this unproven technology, he converts just a portion of the factory — replacing four handlooms with a single steam-powered loom.

The four handloom operators from the bottom row are now in charge of making the steam loom work.

What could go wrong?!

Quite a lot, as it turns out.

1. Skills mismatch. The handloom operators don’t know how steam engines and steam looms work. Though they try hard, they don’t know how to use and maintain the steam loom. They’re weavers, not engineers.

2. Disruption to existing workers. The handloom operators on neighbouring rows really don’t like the noise and disruption caused by the steam loom. It’s distracting and polluting, and affects their productivity and workplace quality of life.

3. Logistical complexity. The steam loom needs a steam engine, and the steam engine needs coal and water to work. Instead of the factory having one input resource to manage — cotton — they now have three: water, coal, and cotton. Because of this additional complexity, the factory needs to recruit another staff member just for handling logistics and building relationships with three different types of supplier.

4. Contamination. It’s quickly identified that coal shouldn’t be kept near cotton or textiles: it makes them dirty, harder to work with, less valuable, harder to sell. Because of this, even more of the factory has to be reconfigured to keep the coal separate from the raw and finished product.

Because of these and related issues, the degree of disruption and change to the factory turns out to be far more than the incrementalist factory owner initially assumed:

Either the factory owner would have stopped the experiment before more than half the factory got turned over to meeting the needs of the steam loom, or they’d have reached this point — and realised that, after all this cost and effort and expense, the net result of all this disruption is lower output at greater cost.

From the factory owner’s perspective, they’d done their due diligence. They adopted an appropriately cautious, experimental and incremental approach to investigating whether the new technology was right for them, and concluded — with strong evidence — that it was a waste of time and resources.

What did the factory owner do wrong?

This is the can’t-get-there-from-here dilemma. The only route from the old way to the new passes through a valley that looks like evidence the new way doesn’t work.

The factory owner is stuck at a local optimum: the best achievable outcome given their current setup. Every incremental step away from it makes things worse. But there exists a global optimum — a far more productive configuration — that can only be reached by starting from a blank page, not by incremental modification of what already exists.

The upstart advantage

Now imagine what would happen if, instead of following the incumbent’s journey, we follow the upstart’s.

The upstart, in normal times, begins with all the disadvantages: no existing expertise, no machinery, no capital, no customers. But in abnormal times — when there are genuinely disruptive technologies — the lack of expertise, resource, and path dependence all become advantages.

The upstart can start with a blank page. An empty field. They can decide how to design their production facilities around the new technology. They can choose their factory location, design the plant layout, hire employees with the right skills and roles — all built from the ground up for the new paradigm. They are not bound by the need for piecemeal incrementalism. They have a tabula rasa.

With the newcomer:

• The factory is located with all three resources — coal, water and cotton — in mind.
• The “dirty” part of the factory is separated from the “clean” part, meaning less contamination of the product.
• There may be fewer employees, but they are more specialist and better paid.
• Every element of the operation is designed to work with the technology, not around it.

The valley between peaks

Let’s imagine the productivity of the two factories over time:

Both incumbent and newcomer adopted the same new technology. But for the incumbent, this led initially to declines in productivity, and then — if they persisted — only to marginal gains. The newcomer also adopted the new technology, but built their entire workflow, logistics, and infrastructure around it. As a result, they were eventually able to achieve far greater productivity than the old factory ever could.

This is the valley between peaks. The incumbent stands on one peak and looks across to a higher one, but the path between them descends into a valley that looks — to every rational metric — like failure. The incrementalist quite reasonably turns back. The newcomer never enters the valley at all: they start building on the other side.

Over time, the industry itself becomes dominated by modes of production based around the new technology. But this doesn’t tend to occur through successful adoption by incumbents. Instead, it occurs mainly through the newcomers — native to the new technology — outcompeting the incumbents by building their entire mode of production around it.

So what can the incumbent do?

If experimental incrementalism won’t work, incumbents aren’t entirely without options:

• Build a pseudo-newcomer. Divert some profits to constructing an entirely new operation — new facility, new staff — who, like the newcomer, start with a blank page and a clear vision of the new technology’s potential. Don’t bolt the new onto the old. Build the new next to the old.

• Buy out the newcomer. At some point, it should be clear that a newcomer will become a highly disruptive competitor. But so long as the incumbent acts quickly enough — and knows the difference between a newcomer who is genuinely using the new technology effectively versus one selling hype — they can acquire the newcomer before they get too big. This means paying generously, based on expected future productivity rather than currently realised productivity.

Both options require something the cautious incrementalist finds deeply uncomfortable: spending significant resources on something that isn’t yet proven by their own experience. But that discomfort is itself the trap. The incrementalist’s evidence is real but misleading — it tells them the truth about the hybrid, while obscuring the truth about the technology.

Which of these options is your organisation pursuing? ¹