agentenv: a local process for talking to agents

Motivation

For about a year my rule for using AI was to put it anywhere a human would otherwise have been needed. I had a long list of ideas built on that rule, and I described them to myself as injecting intelligence at various points of my workflow. The rule is easy to apply and it produces a predictable kind of system: an existing pipeline, unchanged in shape, with a few model shaped holes cut into it where the tedious judgment used to sit.

In September 2023 I heard a better rule on a podcast and swapped mine out for it within a paragraph.

Adjacent work: the control layer frame

The phrase is Marc Andreessen's, from an episode of the Jordan Harbinger podcast, and what I wrote down was this:

""AI will be used at the control layer at every point" -- they were discussing how important governance of models like what information should be included in the models' memory. Marc stated that since AI would basically be included at the control layer of every interface, like anywhere where there is a simple algorithm like a decision tree, it could be replaced with AI. I had previously thought "anywhere where you could use a human," but this "control layer" description seems to be more succinct and accurate as a guiding principle."

The claim in its recorded form is narrow and specific. Wherever a program currently makes a small decision using a simple procedure, a model could make that decision instead. It is a claim about substitution at existing branch points rather than about building something that supervises the whole system.

What makes it a better guiding principle than mine is a difference in what each rule finds when you go looking. "Anywhere you could use a human" finds tasks, and tasks are rare, because most of a working system is not doing anything a person would recognize as a job. "Anywhere there is a decision" finds branches, and any system of reasonable size is mostly branches. They are already written down, as conditionals and thresholds and sort orders, and every one of them was a place where somebody encoded a judgment as a rule because a rule was the only thing available. Once you can put judgment where a rule used to be, the number of candidate sites goes up by orders of magnitude, and they are all already located for you.

The model as a component

Everything downstream of the control layer idea depends on what you think the model actually is, and I wrote my working definition down in the same entry:

"currently, LLM models are "advanced autocomplete", and their main modus operandi is to continue the piece of text which they are given. This means, you can find clever ways to insert information through the customization of the prompt."

A machine that continues text has no memory of you and no state between calls. Everything it appears to know in a given moment is something that was placed in front of it in that moment. That one fact decides the entire architecture of anything built on top: the interesting engineering is not in the model, it is in what you assemble and hand over, and how you decide what to leave out.

The process model

By November I had written down the concrete form, in a task list, on the day I finished moving to San Francisco:

"build "agentenv" — virtual environments for AI agents with a variety of host environments

• a slightly more generalized system for implementing what I actually want — some process on my machine that I can access via CLI / web / REST API for interacting with AI agents. Basically I can tell an agent to do something.
• first environment — local server on machine"

The word doing the work is process. An application is something you open, use, and close, and while it is closed it is not doing anything. A process is running whether or not you are looking at it, which is the difference between a tool that assists you when summoned and a layer that is present at the decision points. Three front doors are named, a command line, a web interface, and an HTTP endpoint, and all three reach the same running thing rather than three copies of it. That is a description of a daemon with several interfaces, and it is roughly what I use every day now.

I would also point out the second bullet against myself. I described agentenv as "a slightly more generalized system for implementing what I actually want," and then stated the actual want in the next clause. The generalization was the part that never got built. The specific thing underneath it is the part that eventually existed.

Choosing the interface

The interface question got answered with a list of capabilities and one cost, which is the shape an honest comparison takes:

"- Telegram as the preferred chat interface

• ability to have buttons
• unfortunately, no streaming chat and limited formatting options
• ability for your bot to ping you rather than you initiating the chat
• ability to have a channel with multiple GPTs having shared common conversation history
• can be generalized further"

The second capability is the one that mattered and the reason is structural rather than convenient. A chat interface where you always speak first can only ever be a tool, because nothing happens until you decide something should. A system that can start the conversation can act on a schedule, on an event, or on something it noticed, and only then is it sitting at the control layer instead of waiting below it. Everything else on that list is ergonomics. That one is the architecture.

Delegating judgment to the model

Two ideas from the same November document look unrelated. One is about deciding whether something is true, the other is about naming what an interface should let you do, and underneath they are the same idea.

The first came from sketching a live translation system. Translation cannot be done word by word, because phrasing later in a sentence changes what came earlier, so something has to decide when enough has arrived to translate:

"given the previous context window and translation, and the current buffer, see whether there is a "sense of completion" — might use GPT to ad-hoc what "sense of completion" means as it is difficult to define but easy to identify."

Difficult to define but easy to identify is the best short description I have of what these models are for. A great many yes-or-no questions fall in that category. Whether a paragraph is finished, whether two bug reports are the same bug, whether a message needs a reply today. You cannot write the rule, and you can recognize the answer instantly, and until recently that gap was where automation stopped.

The second idea makes the same observation about the things a system lets you do, rather than about the questions it can answer:

"I was thinking about how we typically work with accounting using the low-level concrete concepts that are well codified, like a ledger or transaction entries. But humans prefer to think in broader strokes, with concepts. For instance, the concept of "business expense" exists in semantic space as a loose collection of related tasks and conceptual objects, entities, rules, etc.. A Concept-oriented API would expose operations that are also loosely defined."

A ledger has entries and amounts and dates, all crisply defined, and none of them is the thing you actually think about. You think about a business expense, which has no precise boundary and which every accountant nonetheless recognizes. An interface built at the level of the fuzzy concept rather than the crisp record is only implementable if something can hold a loose category, which is the same capability the translation check needed.

The context window as memory

In November a model with a much larger window arrived, and I started treating that window the way you would treat a memory budget. The reason was a measurement rather than a preference:

"The 128K context window doesn't have perfect recall, as evidenced by the needle-in-haystack test performed by that YouTuber.

• Takeaways were mainly: After 64K tokens, performance appears to be degraded for recall. That puts some limits on what I'm thinking.
• Already, 64K tokens is like 80 pages of content — this could potentially be enough total context for just loading an entire repository inside the context."

So the advertised capacity and the usable capacity were different numbers, and the usable one was about eighty pages. That is a resource with a size, filled by decisions about what goes in it, and the conclusion I drew followed directly:

"We could play around with designing / allocating portions of the text context as a sort of "buffer" strategy to work around limitations."

Allocating regions of a fixed space to different purposes, and deciding what to evict, is memory management. It is not a metaphor for memory management. The window has an address space and a capacity and a cost per occupant, and the fact that its contents are English rather than bytes changes the encoding and not the problem.

The compute complexity question

In the same note I asked something I still think is a good question, and gave myself a provisional answer:

"do LLMs act like machines in which more complex problems translate to a higher compute complexity cost? At least for human concerns, or human philosophical "difficult" / vague prompts — it seems to output things linearly ⇒ implies that it's not performing computation / thinking outside applying the training weights."

The observation is available to anybody: ask a hard question and an easy one, and the tokens arrive at the same rate. From which it follows that difficulty, as a person experiences it, is not costing the machine anything, and so whatever it is doing is not the thing we mean by working something out.

I wrote that at the end of 2023 and it was a reasonable read of the machines that existed then.