Cognitive Blocks

Motivation

In the spring of 2023 the only way to get a language model to do something was to have a conversation with it. You wrote a message, it wrote one back, and whatever it knew about your problem had to fit inside that exchange. This works well for things a competent person could do in one sitting and stops working the moment the task is larger than one sitting, which is most of the tasks worth automating.

The obvious response is to use many calls instead of one, and by 2023 there was tooling for that. LangChain was the tool I reached for, and I described it at the time as the best available thing for wiring model calls together. The idea is that you build a chain: the output of one call becomes the input of the next, with code in between deciding what happens.

I wanted to know what the right pieces were. Not how to wire two calls together, which is easy, but what the parts are called when you have thirty of them and they are supposed to add up to something. I called the project Cognitive Blocks, and it never became software. It stayed a design in a set of documents, which is worth saying at the start so nobody goes looking for a repository that does not exist.

Adjacent work: LangChain

The first mention is a side note at the bottom of a journal entry about improving Obsidian's note graph. I was designing a feature that would need some agent process behind it, noticed I did not want to build that on LangChain, and wrote down a framework idea instead.

I should be accurate about why I did not want to. My note gives the reason and it is not a criticism of the tool:

The reason I feel iffy is because I haven't built too much stuff with LangChain and don't feel proficient enough, but this just needs to be fixed by spending more time on some ideas and playing around with it.

Later I did have two specific complaints, and they are worth separating from that. The first is about getting started: too many separate concepts to learn at once, too many tools, and not enough guidance about which to use. I wrote a note to myself in capitals not to recreate that experience. The second is about shape, and it is the one that drove the design: the programming style is imperative where I wanted it to be declarative. You describe the steps in order rather than describing the arrangement and letting something else work out the order.

First prototype: an eighty-page book

I picked a hard case on purpose: get the system to write a short book, around eighty pages. This is well past what a single conversation could hold in 2023, so it would force every problem I was interested in.

What I did first is the part I would still do. Before designing any program, I wrote down how a human writes a book. You answer some questions at the start, about motivation, about what the book should achieve, about tone and length, about which ideas need to be clear and what you will be drawing on. You read the source material and take notes. You build a table of contents, and then outline inside it, several levels down, from sections to chapters to chapter sections to paragraphs. You draft, not necessarily in order. You edit.

Only then did I ask which of those steps a program does, which the user supplies, and what the machine version of "read something long and take notes" would even be.

That last question turned into a specific design. The knowledge base would hold a table of note entries, and I wrote down the fields:

note
  source              document name
  text                lines / quotes to cite
  remarks             AI-generated
      notes           what do you want to remember about the text?
      summary         what does the text say?
      context         details to contextualize the quote
      reason          why is it relevant / noteworthy
  tags                one or more, AI-generated or user provided
  embedding           numeric vector, for search over the whole note

The interesting field is reason. Asking the model to record why a passage is noteworthy is asking it to write down the thing that is usually lost between reading and using, and it is the one field a search cannot reconstruct later from the other four.

Parallel dispatch

I built the writing end and it worked in the mechanical sense. A ParagraphWriter planned a paragraph and then dispatched a SentenceWriter for each sentence, all in parallel. My note on the result is one line:

However the output kind of sucks

The diagnosis took no time, because the failure is a direct consequence of the architecture. Parallel means each sentence was written without knowledge of its neighbours. A paragraph is not a set of sentences, it is a sequence where each one is shaped by what came before it, and I had thrown away exactly that information in order to get the parallelism.

The fix I wrote down does not restore the lost information so much as decide, per sentence, which part of it that sentence actually needs. Rather than giving every sentence the whole outline, give each one a context typed by its position in the paragraph:

The sentence's job in the paragraph determines what it needs to know. That escalated, in the same paragraph of notes, into wanting a small language for describing writing that the model could use internally, where a sentence is an object rather than a string:

SentenceOutline
  sentence type            intro, body, ending
  rhetorical classification  persuasive, explanatory, metaphor

Once sentences have types, a quality process can judge candidate paragraphs and select the best one, which is the editing pass a writer does and which a parallel dispatch had no place to put.

Primitives and connection types

The document states what I wanted to know as two questions rather than one, and the split is the part that matters:

what is a good elementary framework that lets me describe AI agents as composite structures of smaller subagents?

• what are the primitive, atomic structures (leaf nodes) that are unitary?
• what are the types of connections / relations between different nodes that determine how they interact with each other?

The first question is the one everybody asks and the second is the one that makes it a real problem. A list of agent types is a taxonomy. A list of agent types plus the kinds of connection allowed between them is a grammar, and a grammar is the thing that tells you whether an arrangement you have never seen before is going to work.

The role vocabulary

My first sketch, back in April, had four parts: an interface agent as the entry point, a task planner to break a request into sub-tasks, a quality-feedback agent to score another agent's output, and executor agents that each do one thing. Next to the executor I wrote that the granularity of responsibility is one of the main parameters that influences everything else, and the sentence stops there, unfinished, which is about right for how well I understood it.

The later version has fourteen roles rather than four. I named it the Conventional Classification, ConvClass for short, and wrote a one-line definition for each:

I framed the list as approximate descriptions meant to guide how you lay a process out rather than as a schema you have to satisfy, with the reader expected to use their own discernment about which roles apply. That framing is the part I am still happy with, because a vocabulary that admits it is a vocabulary is more useful than a type system that is wrong.

Above the roles sits a ProcessFlow, which describes which blocks are present, how they are connected, and the order of execution, in the way that a program has a main function. The property I care about is that a ProcessFlow is itself just another block. It takes an input and returns an output like any of them, so a whole orchestration can be dropped into a larger one as a single piece.

Splitting and combining

The standard shape for a big task is to split it into small ones, run them, and combine the results. Splitting gets all the attention. My note argues that combining is the part that breaks:

we need to consider the synthesis step as its own problem that needs to be planned and combined together

The example I used was a ten-page report generated from summary statistics that do not fit in a single prompt. You cannot simply hand the pile to a model and ask for the report. You need a process that decides how the report should be structured, and then uses simpler writers to produce each piece with the context that piece needs, and then assembles them. Which is the same decomposition again, one level down.

So the reduce step is itself a map and a reduce. That is the sort of thing that is obvious once written and does not occur to you while you are drawing boxes.

Prompts as syntax trees

A prompt should not be a string, and I have come back to that one more than anything else here.

When a compiler reads your code it does not keep it as text. It builds a tree in memory where the structure is explicit: this is a function, these are its arguments, this expression is inside that loop. Having the tree is what makes it possible to transform the program mechanically, because you can move a branch rather than doing surgery on characters.

Prompts in 2023 were strings with variables interpolated into them, which is the text version, with all the same problems. My note proposes representing instructions the way source code is represented, as composable structures, so that a program can build and manipulate the instruction rather than concatenating sentences.

I did not resolve what that language should be for, and the two options I wrote down are genuinely different. It could exist to make English more rigid, pushing it toward the precision of a formal notation. Or it could exist to make English more manipulable by the programs that handle it, which is a question about the tooling rather than about the language. Those pull in different directions and I left the question open.

Context window growth

In November GPT-4 Turbo arrived with a context window of 128,000 tokens, which was a large jump. My note about it is short:

this is a lot of "RAM" and means that I won't need to make that context management engine for a lot of the use cases that I'm envisioning.

I had been planning to build machinery for deciding what to keep in the window and what to drop, and a capability arriving from outside deleted the need for it in most of the cases I cared about. The note does not argue with this. It records the deletion, keeps the rest of the project, and moves the implementation language to Python.

Being able to throw away a planned subsystem the week the reason for it disappears is a skill, and it is easier to write down than to do, because by then you have usually explained the subsystem to someone.

The parts I would defend

What this produced was a vocabulary and a set of questions, at a point in the year when almost nobody had good words for any of it, plus a prototype that showed exactly where the vocabulary was thin. It stayed a design; there is no repository. The parts I would defend now are the two-part question, because asking about connection types is what turns a list into a grammar, and the observation about synthesis, because everyone still spends their attention on the splitting.