Your game is a harness: the engineering playbook for building games with AI agents

What the source published

Kyle Kukshtel, a game programmer of nearly 15 years, gave a 45-minute talk titled “Your Game Is A Harness” at Indie Rodeo in Calgary on September 12, 2026, then rewrote it as a blog post (not a transcription) — which he pointedly notes is 100% human-authored. His subject is using LLMs for real game development: eight hours a day, five days a week, not the one-shot demos that look cool on Twitter. His own journey started before the agentic era, when he blindly copy-pasted code into ChatGPT's web interface to build a moodboarding site, mood.site — it took about 20 hours across 2–3 weeks of evenings, and it worked well enough to convert him.

His core claim is the one he slides as “LLMs are Bad At Games” — but he argues it is a structural problem, not an intelligence one. The one-shot Three.js workflow works because everything is code: an LLM can reason about code, change it, and see the result. Traditional games are full of non-code “stuff” — assets, scenes, and dozens of systems that are not easily representable as code — and those parts are effectively opaque to a model. He also notes that high-quality game codebases to train on are orders of magnitude rarer than web projects. To set up the fix, he explains prompt anatomy (system, assistant, and user prompts), how every message in a session re-submits all prior context, and how that creates “context pollution” — early bad material keeps informing new work — which is what the emerging practice of context engineering tries to manage.

The fix starts with the idea of a harness: a set of action patterns an agent is meant to follow in response to some expected input. Claude Code is already a harness; harnesses can be augmented, extended, and specialized for the work at hand. Studios are building them for everything — browser iteration, telemetry, profiling, tickets, docs — but none of them can actually talk to your game: a game is a black box, and most of what an LLM sees is only the code-as-written, which is not enough. The task, then, is to turn all the stuff in games into something an LLM can work with directly: crack open the black box and make the game legible to an agent. He adds that most of the techniques below are also just good game engineering, supercharged by AI.

Sources: [1] [2]

Try it: turn your game into a harness in six steps

1. Lock down randomness first: from day one, route every non-deterministic call through a single seeded randomness source, so the 30th Random.Float call is the same every time — deterministic state reproduction starts here, not in the last month of production.
2. Build save/load as infrastructure: from the very start, make it trivially easy to save and load an exact game state. Ask yourself how easily you can capture the current state and restore it byte-for-byte; amateur teams defer this to the end, and that is a grave mistake.
3. Close the screenshot loop with diffing: add screenshot capability (most engines have one), and let yourself toggle UI and gameplay layers on and off to reduce visual noise. But don't throw raw screenshots at a multimodal model — image noise is its own context rot — instead diff before/after frames with the free CLI tool odiff, and use the quantitative similarity value (like “85% similar”) as a stable evaluation baseline across agent turns.
4. Adopt Thing-driven objects: replace a hundred-class inheritance tree with one fat struct, Thing — mesh, collider, stats, an isAlive flag, and a type enum. A tree and a dog are, to the code, the same kind of thing with different components. One object type means a single source of truth, so the AI always knows exactly where to change things, and serialization becomes trivial: one way in, one way out.
5. Go commands plus external data: funnel every state change through serializable command objects processed by one class. Commands can be recorded and played back to reproduce bugs exactly, hand-written in a text editor (by you or the AI), and applied to a running game with no compilation — a typed, “puppet-able” interface to your game. Move all static game data out of the code into external sources that get deserialized at load, so tuning a walk speed no longer means recompiling.
6. Isolate the simulation, profile as text: split the game logic into a standalone simulation library that the engine talks to through commands, like a web client talks to a server — the engine stays dumb and just executes whatever state the simulation returns. That lets you test game logic without any engine running, using standard tooling. For performance, skip profilers built for human eyes; use the free tool samply, which dumps captures as plain console text that an agent can read, cross-check against its docs, and iterate on autonomously.

Sources: [1]

The full picture: why each technique helps the AI specifically

The screenshot-plus-diff pairing matters because Kukshtel tested the naive version: even a multimodal model has a very hard time telling what changed between two screenshots, let alone whether the change moved toward the goal. In his pirate-skeleton example, asking to “make the skeleton a pirate” forces the model to process the whole annotated image — everything else in the frame becomes context pollution. odiff outputs both a diff image and a number, and the number is what a model reads well: it becomes a stable, quantitative baseline of “how close am I” that an agent can optimize across turns. Combined with cropping, you can make the model look only at what is supposed to change.

Thing-driven development is, in his words, what “AI really likes.” With 100 classes and 100 object types, an agent must hold all their relationships in context to act, and it agonizes over whether to extend a class, morph a type, or create a new one — context explodes and decisions get messy. One object type kills the ambiguity: there is exactly one place to change, every time. Saving and loading collapse from a nightmare into the trivial: one type, one serialization path, no polymorphic deserialization races or object-tree wiring. A game becomes a giant list of Things; saving is saving all of them, loading is the reverse. He credits similar ideas in Media Molecule's Dreams, discussed on the Wookash podcast with its creators.

Commands and externalized data are about tightening the loop. The command pattern gives the AI a typed interface to the game without reflection or bespoke processing — commands can originate outside the game entirely. His own proof point is Cantata, where about seven non-programmers edited game data through a custom editor called Depot and iterated against plain builds without him cutting new ones; an agent can be taught to use your data tools the same way. Isolating the simulation core goes further: with logic in its own library, you can rip through tests in a pure C# project with standard tooling and never boot the engine — and because the game is already built to consume state diffs, those diffs can come from anywhere: the network, an LLM, whatever. Brian Cronin described this “Blackbox Game Architecture” for Monster Train at Roguelike Celebration, and the Solium Infernum team used a similar approach.

Assembled, the full loop looks like this: deterministic inputs, structured no-compile state changes, verification of each change in isolation, and a performance check — with an agentic turn running the whole thing in seconds. Because the loop is fast, it can run against every change rather than being saved for when it “matters.” His postscript is an honest scaling note: he is shipping a small project that uses none of this, because the target scale didn't need it — these techniques earn their complexity on larger games.

Sources: [1] [2] [3] [4]

Key takeaways

• Core thesis: LLMs are not intrinsically bad at games — it is a context problem. Non-code assets are opaque to models, so a harness is the discipline of making the game legible and directly operable by an agent.
• Determinism first: one seeded randomness source from day one, save/load as infrastructure rather than a late feature; the metric is how easily you can capture and restore the exact current state.
• Don't feed raw screenshots: diff before/after frames with odiff and use a number like “85% similar” as the eval baseline — image noise is its own context pollution, and cropping keeps the model focused on what should change.
• Thing-driven objects: one fat struct for everything, a single source of truth so the AI always knows where to change, one serialization path that makes save/load trivial.
• Commands plus external data: state changes as serializable, replayable, no-compile commands; tuning data lives outside the game so agents iterate with your data tools instead of waiting on builds.
• Isolated simulation: game logic in a standalone library, the engine consuming state diffs — tests run without the engine, and performance is profiled with samply's plain-text output that an agent can read and act on. An agentic turn can run the whole loop in seconds, against every change.

Sources: [1]