#AGIYOURSELF: ninety days of automations, ninety days of prompts

Motivation

In March 2024 I gave myself a challenge. One automation a day, every day, for ninety days, built out of whatever AI tooling I could wire together. I called it #AGIYOURSELF.

What set it off was a gap I could name. Models were getting dramatically better every few months and almost nothing downstream of them was changing. I wrote it down at one in the morning on the second day. If the foundation models are the chips of the AI race, then what is the operating system for development, because the applications are obviously the automations, and there is no common SDK for the working man. When a new chip arrives, people make better games. New models kept arriving and the software layer above them did not move. So I decided to fix it for one person first, and the person was me.

The daily cadence

The reason a daily cadence seemed right, rather than arbitrary, came out of a comparison I still think is half correct.

A habit in a person is expensive. You install it by repetition, it decays when you stop, and on any given morning it can lose to a worse habit that is easier. An automation does none of that. Once it runs, it runs the same way tomorrow, and it does not need to be talked into anything. So building one is like installing a habit that cannot decay, which means every automation you add raises the floor you operate from, permanently, and ninety of them in ninety days should compound into a different person.

I called the idea atomic workflows, borrowing the frame from James Clear, whose line is that you do not rise to the level of your goals, you fall to the level of your systems. Every automation you build upgrades your baseline, and the whole exercise is a way of externalizing habits into an intelligent process.

That model is why the challenge was a daily challenge. Compounding needs a rate.

Capture before meaning

The first thing I built was a pipe and nothing else.

A message goes into a Telegram chat. An n8n workflow catches it and appends it to that day's page in Notion, with Redis in the middle serving as a cache and a message stream. There is no schema, no processing, no classification, and no plan for what any of it is for. I wrote the honest version that night: I have not figured out how to process it later, but at least the capture step is complete.

Deferring the meaning was the right call and it took me months to be able to say why. A schema is a set of answers to questions you have not asked yet. Choose one on day one and you will spend the next year either fighting it or migrating it, and in both cases you are maintaining a guess. Text in date order commits you to almost nothing, and almost nothing is the correct commitment when you do not yet know what you are collecting.

The cost of that choice is that you carry an unanswered question around with you. I carried mine until August, when it came due in one line: I never had one specific goal in mind except to store as many things that enter my mind, append-only style, and I always pushed off the concern of what GPT would do with this data until later, but that later is now.

Error handling

On day two I built a workflow to process my email and did not build an error path for it.

I thought I had turned it off. Turning it off was not what stopped it, and it kept running while I did other things. Around midday I noticed eighty dollars of API credit gone in two hours. By the time I found it properly, the workflow had made about six thousand API calls against a hundred emails and cost roughly four hundred dollars.

The rule I wrote down that afternoon is the one I still use. Test loops against a cheap local model first, and measure the token spend before letting anything run unattended. An automation that runs without you is a program with no bounded blast radius, and the only thing standing between it and your credit card is a stopping condition you remembered to write.

The unit of accrual

By April the cadence had slipped, and the reason turned out to be more interesting than discipline. The automations I had built were quietly stopping being worth having, and I could not point at a compounding effect anywhere. The problem was not that I was shipping too slowly. It was that each thing I shipped was worth less than the last.

The failure is in the unit. An automation encodes a specific task, which means it encodes every assumption around that task: the tool it talks to, the shape of the input, the moment it should fire, and my belief in March about what was worth doing. Change any of those and the automation is not degraded, it is wrong. A habit for a machine does execute perfectly, exactly as the comparison promised. It just executes whatever you told it, forever, including the parts that stopped mattering.

There is also a gate in front of every one of them. To build an automation you must first decide that a task is worth automating, and that decision needs evidence you do not have yet. So I spent April stuck in front of the gate rather than behind it, and I diagnosed it at the time as perfectionism preventing me from working on the workflows because of the need to design them and figure out the utility first.

When I wrote the retrospective, I put the arithmetic plainly: ninety automations is worth less than one automation with ninety refinements.

The substrate had a share in this and it is worth being specific about which share. I built the season on n8n, and by April I wrote that it was getting unwieldy, because a visual workflow becomes implicit spaghetti once it is large enough, and it has no typing and no variables to hold it together. That is a real limit and it is also the price of the thing that made n8n right for March, which is that you can build a working flow in an afternoon without writing a program. I still used it afterwards for exactly that, including the Telegram bot that survived everything else.

I spent a day in LangChain testing whether code would do better, and came out unconvinced in a way I tried to keep honest at the time: it felt overly complex, with an abundance of structure and verbosity, too many disparate concepts, and no happy path, and I wrote in the same sentence that I did not know whether it was a good framework and would give it the benefit of the doubt. That hedge was doing real work. I had spent one day with it. What I could legitimately conclude was about me and my problem, which is that neither a visual builder nor a general-purpose framework was going to fix a project whose unit was wrong.

Prompts as code

The second season replaced the unit. Instead of ninety automations, ninety prompts, kept in a git repository and treated with the same seriousness as source code.

Writing a prompt costs minutes rather than an evening, which removes the gate entirely. You do not have to be sure a prompt is worth having before writing it, and a prompt that turns out to be useless costs you almost nothing, so you can afford to be wrong far more often. That is the whole economic argument, and it is why the second season was easier to sustain than the first.

The mapping I wrote down runs the analogy all the way through. Prompts are the source. Build scripts turn a prompt into a runnable program. Documentation is not optional, because a prompt whose purpose is undocumented is unreadable three weeks later. Execution scripts run prompts over data. And eval suites are the test suites, except that they return scores instead of pass or fail.

The architecture that makes it work is a separation I wrote in capitals because it had taken me a while to get to. Prompt-land is composition: interpolation, iteration, everything about how a prompt is assembled. Code-land is execution: an engine that injects the context and renders the template. Prompts do not belong in Python code files.

The reason that matters is mechanical rather than aesthetic. A prompt built by concatenating strings inside a function cannot be diffed as a prompt, reviewed as a prompt, or tested as a prompt, because to a version control system it is a change to a Python file. Pull it into its own file and it acquires a lifecycle: it has a history, it can be reverted, and someone can review the change to the wording without reading the code around it.

In the repository the split is the directory listing, which is the shortest way to see it:

promptbase/          prompt-land
  extract-info.j2
  analyze-convo.j2
  library.lib.j2
  journal/
    compare.j2
    extract-ideas.j2

agiyourself/         code-land
  cli.py
  promptfile.py
  chat.py
  journal.py

No prompt text lives on the right. No execution logic lives on the left. A prompt is a file with frontmatter and a body, and it looks like this one, which pulls the fields it should extract from whatever context the engine hands it:

---
name: extract-info
tags:
  - utilty
---
Extract the following information from the input:

{% for key in keys %}
- {{ key.name }}: {{ key.description }}
{% endfor %}

Input:

{{ INPUT() }}

The typo in the tag is in the file and I have left it there, because a promptbase accumulates the same way a codebase does.

What made the analogy pay rather than merely hold was library.lib.j2, a file of macros that other prompts import. Chain of thought, role prompting, and structured output are each written once as a function taking arguments:

{% macro role_prompt(role, expertise, task) -%}
Assume the role of a {{ role }} with expertise in {{ expertise }}.
Your task is to {{ task }}.
Approach this task from the perspective of your role, utilizing your specific
knowledge and skills.
{%- endmacro %}

That is the moment prompts stop being text you copy and start being components you call. Improving how role prompting is phrased becomes one edit in one file rather than a search across everything you have ever written, which is the ordinary benefit of a function and the exact thing the first season never got.

The tooling followed from there. Prompts were Jinja2 templates, chosen over Handlebars because the template language is more extensible, with blocks for inheritance and macros that behave like functions. Two components got names, a prompt language and an execution engine. The command line tool got three verbs working, run, list and info, with search written into the spec and not built, and the version running in July printed hello world.

Evaluation: scores instead of pass or fail

Scores instead of pass or fail is the whole difference between testing software and evaluating a model, and it is the single line I would keep from all of this.

A test asks whether a function returned the right answer, and the answer is yes or no. There is a correct output and the code either produces it or does not. Ask the same question of a prompt that summarizes a journal entry and it falls apart, because there is no single correct summary, and two summaries can both be reasonable while one is clearly better. What you can measure is how good the output is along whatever dimensions you care about, which is a number, not a verdict.

Once your check returns a number, you can compare two versions of a prompt, and once you can compare versions you can improve deliberately instead of by feel. Without it you are editing wording and hoping, which is what most prompt engineering actually is.

That leaves the question of what you score against. Public benchmarks cannot help, because they measure performance on somebody else's problems, and my prompt for extracting what mattered from a day is only good insofar as it works on my days. I had three or four months of journal entries sitting in a database by then, which is the one dataset in the world that is exactly on-distribution for the tool I was building. So the eval suite was the journal.

Designing for incompleteness

In September I wrote down the principle the whole thing had converged on, and it is the part I would give somebody starting today.

Automations in this era should not be designed for efficiency first. They should be designed for correctness and learning, to keep as much optionality and ability to pivot as possible. The reason is that the ground moves: the model underneath your system is changing faster than your system is, and a design optimized for how things work this quarter is a design you will be unwinding next quarter.

The operative half is the assumption you build in. Whatever system you build has to be robust enough to collect data while assuming that the data will be incomplete. My automations were incomplete, unreliable and going to break, and the ones that survived were the ones that kept collecting anyway.

What the promptbase accumulated

Months later, thinking about something else entirely, I worked out what the promptbase had been accumulating, and it was not prompts.

Looking back at the prompts I had written, the thing they had in common was not the wording and not the technique. Each one had come from a situation that kept recurring in ordinary life, and what I had really been recording was the trigger: the moment when I would want this, and the intention behind reaching for it. The prompt was the cheap part. Knowing when it should fire was the part I had spent ninety days learning.

That is the third answer to the same question. I started by collecting automations, moved to collecting prompts, and what accrued underneath both was a catalogue of the moments in a day when a system could have helped, which is the thing you cannot get by deciding in advance and can only get by keeping the pipe open and paying attention.