Banning the word 'context'

For about a week in April 2025 I could not describe the system I was building, and when I finally worked out why, the cause was a single word that I had been using to mean four different things.

The system was meant to run multi-step AI processes: read a document, ask a model something about it, take the answer, do something with it, feed the result into the next step. The question I could not settle was what travels between those steps. A block of text is the obvious answer and a bad one, because the next step has to parse it back into something usable and parsing prose is where these systems break. JSON is the other obvious answer and it is worse for a different reason, since the thing being passed around is often a document that a person also needs to read and edit.

So I had a placeholder called IdyllicValue, which was whatever the right answer turned out to be, and every time I tried to specify it the specification came apart in my hands.

Motivation

It was not that I had no ideas. It was that every path forward opened into more paths, and holding them all at once was the thing I could not do. My own note from the middle of it:

the idea is that we want to keep it simple (keep it primarily agentic) but have clear semantics about how context is implicitly transferred. i think this is what i'm missing clarity on, because if we want the agentic process to potentially work for 20-50 steps the context between should be light? so i was thinking just references, but fuck when I think about this i can sense it getting extremely cloudy again and exploding into superposition of possibilties

Superposition was the word I kept using for that state, and it is the right one. Not confusion, which implies you have the wrong answer. A superposition is many answers held at once with nothing yet forcing a choice between them, and the sensation is of the problem getting larger every time you look at it directly.

Read that note again and notice what is doing the damage. "Clear semantics about how context is implicitly transferred." Every load-bearing word in that phrase is vague, and the vaguest is the one I was leaning on hardest.

The overloaded term

Context, in that sentence, was standing for at least four separate things.

It meant the environment a step runs in, which is to say the names that are visible and the values they hold. It meant the specific document that had been resolved and pulled in for this step. It meant the request eventually assembled and sent to the model, which is a different object with different rules, since it has a size limit and an ordering the environment does not. And it meant the state of the running process, which persists across steps while the other three are rebuilt each time.

Those are four things with four lifetimes. You cannot give a type to a word that means four things, so the type system stalls until the vocabulary does. What felt like an architecture problem was a naming problem wearing an architecture problem's clothes.

The fix was blunter than the diagnosis. Rather than defining the word more carefully, I banned it:

let's decide to not use the term "context" in any form in order to be more clear, please offer precise alternatives

Not "let us be more careful with this word". Remove it from the vocabulary entirely, and require that anything it had been carrying be renamed into something specific. A word you are allowed to keep will keep absorbing whatever is nearby, because that is what a comfortable word does. Taking it away forces every use to declare which of the four things it meant, and the ones that cannot declare were never saying anything.

What came back was a proposed set of four names. Accepting a name and accepting it for a reason are different acts, and my reply did both:

1. environment / scope
2. resolveddocument
3. llmrequest (cuz we can put things like provider / temperature / tools)
4. ExecutionState (it feels like environment -- but we could always merge later)

Two I took as offered. The third I renamed, and the parenthesis is the whole argument: the thing being described is not a payload of text, it is a request with provider, temperature and tools hanging off it, and a name that says payload will keep hiding those. The fourth I accepted with a doubt attached, that it might turn out to be the environment under another name, which is worth recording rather than resolving early.

The four have four lifetimes. An environment is rebuilt per step, a resolved document is fetched per step, a request is assembled and sent once, and the execution state outlives all of them.

I said at the time that clearing up ambiguities is a hundredfold amplifier, "the same way we don't use the term agents or memory," and I would still recommend the move generally. When a design will not resolve, check whether one word in the specification is naming several things, and if it is, ban it for a day and see what has to be invented to replace it.

Reducing to one type

The other move that month was to stop having many kinds of thing.

For the previous three months I had been building semantic objects, which are typed containers the model holds a reference to rather than reading. A journal collection, a blog post, a health record, each with its own fields and methods. It worked, and the cost of it was that every new use wanted a new type.

In April the whole zoo collapsed into one:

the document is going to be the first and only semantic object we focus on, perhaps later we'll bridge the concept but this is the only container we need to focus on even if generic because I think we can just bring our wishes into the documents (imagine defining custom functions on the document etc)

The clause in the middle is the reason it works. The argument for many types is that different things need different operations, and a journal collection genuinely does need different methods than a blog post. But if a document can carry its own functions, then the variety moves inside the container instead of living in the type system. One generic thing that can hold custom behaviour covers what a dozen specific things were covering, and nobody has to design a type before they can start.

That is a simplification rather than a retreat. From the outside "we deleted all our types" and "we gave up" look identical, and the difference is whether the variety went somewhere or just went.

Specifying the container first

With one container to design, the next move looked obvious: specify what the container guarantees.

The session produced a list of eight properties an IdyllicValue should have. It should be structured rather than plain text. It should carry citations back to its sources. It should be readable by a model and by a person, both. It should be composable, metadata-rich, editable, and operational, meaning it could carry things that execute rather than only things that describe.

The plan was to take one property a day and build the smallest demonstration that it was real.

I abandoned that plan within the same session, and my note about it is the most useful sentence I wrote that month:

I don't know what the fucking IdyllicValue structure is, we have to figure this out by necessity therefore it can't be the fucking first thing we do, it's a superposition to collapse

The reasoning underneath is not impatience. A data structure is determined by the operations performed on it. If those operations do not exist yet, then specifying the structure means guessing which operations will matter, and you will guess wrong in the specific way that is hardest to detect, which is that the structure will be exactly good enough to build the things you imagined and awkward for everything you did not.

The same session has me applying the test to my own work in the bluntest available terms:

what are we trying to prove here, structured output without json? nothing here really screams "single LLM call couldn't solve that" why do we even need this parseability

That is a good standard and I have used it since. If a design cannot show something a single model call could not have done, the design has not yet earned its complexity, whatever its properties list says.

Reframing to operations

By the evening the question had changed shape, and the new version was better because it was about verbs rather than nouns:

instead of thinking of the 8 properties of IdyllicValue, let's think about 8 properties of documents as computational substrate. what is necessary? and then let's think about 8 tools that we want to give AI to manipulate. for example adding annotations is KILLER feature because it opens up automating annotations as a UX primitive. like, what if I want to create a system that auto-grades my korean lesson quizzes?

"What data structure do we pass" is a question you can answer wrongly for a long time without noticing. "What can an AI do to a document" produces a list you can start building on Monday, and each item on it fails visibly.

Annotation was the one I kept coming back to, and the reason is that an annotation is bound to a specific block of a document that a person is looking at. The editor underneath was BlockNote, which matters here for one property rather than as a stack detail: it models a document as a list of addressable blocks rather than as a stream of rich text, so a block already has an identity a comment can be attached to. The primitive I wanted was available because someone else had already made the document addressable. A model that can attach a comment to a paragraph has a way to say something that lands in the place it is about, and a person can see it, disagree with it, and delete it. That makes automated commentary into an interface rather than a feature. The Korean quiz grader is the smallest complete version: the quiz is a document, the grading is annotations on it, and the thing I would actually want to use exists as soon as those two work.

There is a cost to making documents executable that I noticed in the same conversation. If a document can contain instructions and an agent reads documents, then anyone who can write to a document can instruct the agent. Homoiconicity, which is the property of a system where the program and the data have the same form, is a lovely thing to have and it hands you a prompt injection surface for free. Any primitive that generates from a document has to be scoped with that in mind, and noticing it early was worth more than the eight properties were.

What it cost

Two things, and neither is the value type.

The first is the vocabulary move, which is now a standing habit. Ban the word that is doing too much work and see what has to be invented to replace it. It is cheaper than an architecture and it fixes the same class of problem, because a specification you cannot write in precise words is not a specification you have.

The second is the ordering rule that came out of abandoning the properties list. The structure of the thing passing between steps could not be designed in advance, because it is determined by operations that had not been built. I had made this mistake before, on a compiler, where I spent a year designing an intermediate representation without having written the code it was supposed to produce. Both times the fix was the same: build the concrete case, then let the shape fall out of it. Learning it twice, in domains that look nothing alike, is how I know it is about design rather than about compilers.