The Idyll

Motivation

Most of the work being done on agent systems is solving one of two problems. The first is reducing how much a human has to prompt, by automating the prompting, obviating it, or abstracting it away. The second is giving a human a legible surface to steer from, so that the higher-level work can be navigated instead of typed.

Idylls came out of asking what I was actually doing while running a lot of agent sessions at once. The answer is that I was the convergence layer. Some people call this taste. Whatever it is called, it is manual, and it runs on me. I hold the context, I hold the goal and how close we are to it, and I hold how the pieces relate to each other, which is an index that exists nowhere except in my head.

That last part is the one that matters, because it is not a memory problem. The destination is not missing from the model. It is missing from everywhere. I carry it between sessions, and every time I get pulled into steering the model through some detail I put it down and pick it back up slightly changed. Prompt libraries, skill packs, and the various ways of giving a model more context all improve the model's side of that arrangement. None of them touch the part where I am the only copy of the map.

An idyll is the map, written down outside your head, in a form that something else can check against.

The infinite-context test

Most of what I built in the first few days did not survive, and what removed it was a single test I can state in one sentence.

Assume infinite context. Every session, every revision, every message, every tool call available to every future computation, at no cost. Now ask what is still useful.

Two things survive. The articulation, which is the destination and its conditions, because that is supplied by a human and cannot be derived from the record. And the verification, which is the act of checking those conditions against the record, because that is a computation over the record rather than a summary of it.

Everything else I had built failed the test. The state summaries, the attempt indexes, the registers, the readings: each of them is a smaller thing standing in for a larger thing so that you do not have to read the larger thing. That is a cache.

A cache is a fine thing to add once you have watched the system run and know which questions get asked often enough to be worth precomputing. It is a bad thing to design in advance, and worse to build into the shape of the object itself, because at that point it has stopped being a shortcut. It is a decision about which parts of the record matter, taken before you had any evidence about which parts matter, and everything built afterwards inherits it. Premature compression was the failure running through everything I designed that week, and assuming infinite context is what catches it before it gets built.

The most concrete casualty was scoring. I had a distance number and a convergence verdict, both of which felt like the obvious way to show progress. Both are lossy caches of verification, and a model asked to produce one will produce one whether or not it can. What replaced them is narrower and duller and much better: progress is whether each written condition is satisfied, checked one condition at a time, with the evidence cited, and with a third answer available besides yes and no. A condition can be not verifiable, and saying so is a real result.

The folder

An idyll is a folder. That is close to the entire idea, and I want to be careful not to make it sound like more than that, because the whole point of the week that produced it was discovering how little needed to be built.

Inside the folder there is a file describing the destination: one paragraph of what the world looks like when this is done, and then the acceptance criteria, each of which has to carry its own measurement procedure. There is a separate file holding the current position, with its own date on it, kept out of the destination file so that where you are can never quietly become part of where you are going. There is an append-only list of what has been tried and what it yielded. And there is a folder of dated increments, the contact with reality: the things that happened, the reactions, the evidence.

Written out, the whole object is five entries, and the only one whose internal grammar varies is the destination file.

IDEAL.md      destination: one paragraph, the state of the world when this is held
              acceptance criteria: each carrying its own measurement procedure
              (domain section): the grammar varies HERE and only here, as
              policies-under-test | rules-with-causes | hypotheses | pipeline-stages
              | falsifiers | curriculum, one per domain family
STATE.md      position, as-of dated, glanceable, never mixed into IDEAL
ATTEMPTS.md   tried -> yielded (worked / didn't / unclear), append-only
contact/      dated, object-tagged, polarity-typed increments with excerpts
registers     standing records for recurring entities (people, policies, incidents)

The variation is confined on purpose. A weight goal and a job search and an essay need different things in the middle of the destination file, and they need identical machinery around it, so the part that differs is fenced into one section and everything else is fixed.

None of that is written into the program. The code has no catalogue of goal types, no fixed list of stages a goal must move through, and no notion of a verdict at all. Everything that differs between a weight goal and a job search lives in the text of the files, where a person can read it and change it with an ordinary editor. The code knows only that there are folders and that folders have files in them. That division is the rule I ended up writing down, and it is what keeps the domain out of the software.

Deriving the format from twelve folders

The part of the week I would repeat in any other project is the way the format got decided.

I did not specify it. I built twelve of these folders, seeded them with real material out of my own record, and ran a custodian over them: one model call per pass, no tools, folding each new increment into the state and reporting back. Thirty-two passes in total. Five competing grammars for the destination file, plus two controls, one that received nothing but a quiet period and one that started with no contact at all.

Every pass returned the same five things, and it is the last one that paid for the experiment.

distance       how far position sits from the criteria, or an honest refusal
error signal   the next thing that would have to be true
verdict        converging / not converging, with evidence
ideal-changed  whether the destination moved this pass, and why
missing:       what the structure lacked in order to conclude anything

The first four are the custodian doing its job. The fifth is the custodian reporting on its own working conditions, and it is the one that turned an experiment into a specification.

Thirty-two of those complaints, sorted by frequency, are a specification. The top one was reported in more than ten passes across every domain, and it is embarrassingly mundane: put absolute dates on everything. A contact line stamped with a clock time and nothing else makes every derivative quantity uncomputable. You cannot get a rate, a streak, a weekly count, a staleness, or a distance to a deadline out of a stamp that does not say which day it was. I had been writing lines like that for years in a format I liked, and no amount of thinking about the design would have surfaced it, because it only breaks when something tries to compute across the lines.

The second was that the work itself has to live inside the folder. When an idyll referenced its artifact somewhere else, the custodian could describe progress and could not check it. The phrase in the report is that it degrades from verification to hearsay, which is exactly right: without the artifact in hand, all the custodian can do is repeat what the record claims.

Kinds of destination change

The other result I did not expect is that changes to the destination file come in three kinds, and they mean completely different things.

Sometimes a criterion gets sharpened without the destination moving: a vague condition becomes a measurable one. That is healthy, it happens most at the start, and it should get quieter over time.

Sometimes the destination itself is understood differently, because you have learned something. That is the actual learning the object exists to hold, and it should be shown to you.

And sometimes the record reveals that you have been converging on a different thing than the one you wrote down. One of the twelve caught this live: its stated destination was one kind of probe, and every piece of contact in its folder was about a different one. It said so, and re-aimed.

The handling rule falls out of the distinction. The first kind can happen silently. The second should be surfaced. The third has to stop and ask you, because it is not a refinement, it is a report that your map and your walking have come apart, and no machine should resolve that on your behalf.

Convergence detection

None of this gives you a progress bar, and I stopped wanting one. What the lab produced instead is three trends that have to agree before you are allowed to say something is converging.

Position moves toward the criteria. The criteria stop moving, with the churn decaying from the third kind through the second to the first and then to silence. And the error signal gets narrower and cheaper, from "figure out how to measure this at all" down to "make this one edit".

The compact version is that an idyll is converging when position approaches criteria that have stopped moving, and each attempt buys a narrower next question. All three are computable by the custodian, and none of them require the human to have an opinion.

Divergence has its own smells and they are just as legible: flat distance with the same error signal repeating, destination churn of the third kind, and contact arriving that never changes what has been tried, which is motion without information.

What the markdown test returned

The software worked. The build record at the time lists 87 unit tests, 23 behavioural passes including live model turns, and 41 design locks that a build had to satisfy. Seven open design questions had been asked one at a time and answered.

I shelved it anyway, on a branch, and shipped the flow as a skill file plus a standalone verifier you can point at any directory that contains a destination file.

The reason is a second test, and this one came back negative. If you cannot prototype the concept in plain markdown, you do not have the concept yet, and I could not get it simple enough to express that way. Twelve folders run by hand did not produce a shape a person could write down without the machinery around it, and every grammar I tried to give them added structure I could not justify from the data I had.

So the honest reading is that the idea was not mature. I had run it for a week, which is enough to find out that I was the convergence layer and not enough to find out what that layer actually does. What I need before picking it up again is more of the thing I was short of, which is sessions run against real destinations, so the shape can be derived from them instead of proposed at them. It is on a branch, and I will know it is ready when I can write it out in prose and it holds.