Delegate Decisions, Branch for Information
Delegate Decisions, Branch for Information
— 5 notes on agent harness building
1. To be the right amount of AGI-pilled
Do not ask an LLM to do the stuff it was not intentionally trained to do, such as performing human emotions. Even if you fine tune the harness to make it happen, the model won’t necessarily perform it well, and you have to do it over every time a new model is released.
It’s a bitter lesson from the past few months. LLM providers intrinsically have a bigger say in what future agents will look like. It’s vital to know the model you’re building with, and to know where the new model is headed in the near future, which is exactly what Cat Wu said about being the right amount of AGI-pilled.
2. Do not compare an agent to a human
An agent itself has no memory. It is wrong to compare an agent to a human when it comes to understanding what an agent is.
An agent is a class — prompt, tools, skills. Anything that doesn’t initially come into the agent’s system prompt is decoupled from it, such as memory as a mounted file system.
Agents running with the same class are different instances, but they’re actually the same agent.
The focus diverges from there: class-centric or instance-centric. It determines what your agent product looks like.
If you choose to focus on the agent instance, you’ll struggle with problems like how to mimic humans’ memory effects, which is how to persist valuable information that appears in the context window, how to effectively store it in a persistence system such as in a file system, and how to retrieve it as the user expects, etc. All of these are difficult to solve, and some of them have less to do with the agent itself.
I believe these are distractions for agent technology development. It is wrong to pursue the ultimate goal of building a human-like agent from sci-fi movies — a wish that naturally pops up in the minds of those who know little about LLMs and agents. The distraction isn’t persisting information; it’s persisting it to make the agent feel human. Storing the inputs a decision actually needs and mimicking human memory happen to use the same file system, but they are not the same goal.
3. The context window is the only key constraint, and it forces recursion
Context window is the only true core constraint that governs how to design agents. The limited context window is the hard limit of LLM as a technology.
Based on The Nature of Technology by W. Brian Arthur, the hard limitation will lead to recursion if we want to do more with AI agents.
Here are two mainstream directions on how to recurse:
The first way is the Claude Code dynamic workflow feature. All the agents invoked in workflows are more like tools. The main session delegates those decisions that have already been well-trained into models so that they can be completed perfectly and has no reason to take up any room in the limited context window of the main session.
The second way is the Slock (rebranded as raft.build now) way. The recursion with Slock happens upon the agent instances rather than agent classes. The key difference is that those agent instances act like members making decisions that can’t be made only with LLM inner knowledge separately because each of them has a mounted file system as memory store and it stores information from users, not from models.
4. The context-centric division principle: delegate decision-making, branch for information-pulling
The principle is one line, two halves:
Delegate when the other agent must own the decision. Branch (query the duplicate) when you only need to read from it.
Everything else that crosses the boundary is pollution.
In the Claude Code dynamic workflow mode, non-general decisions are all made by main sessions. When different questions from various decision domains pile up, they pollute each other given a limited context window.
When it happens, it’s the right time to separately store the key information needed to make those non-general decisions, and make sure it does not happen in the same context window.
Slock happens to partially achieve this effect, but it wasn’t building towards the vision by intention. So the interactions between agent team members may not be 100% necessary. When it’s not, the pollution still happens.
5. The bottleneck is the information trapped in human heads — Toward self-speciating agents
When I started building the automated issue pipeline for team, it started as a simple loop. It ran one tick every five minutes through the day — fetch, ingest, sweep Slack, triage, open the fix PR, housekeep. Six steps, fully looped.
As I kept building it more autonomous, the biggest obstacle was no longer the agent, it was the waiting for human colleagues to make decisions - what to do with each issue.
The reason why it was not autonomous is that the information needed to make high-quality decisions is not in the codebase, but in human colleagues’ brains. Even if we have many docs, decision-making still relies heavily on humans because some key information is not preserved in the docs, and we naturally hate writing and updating docs.
The most frequent thing I did in the past weeks wasn’t coding, wasn’t building features with Claude Code. It was replying to Lindsay (my agent in charge of the issue pipeline) with the decisions I made, and the logic behind each one.
This was a good signal to do the separation. It constantly asks for the same type of information, and it was decoupled from the triage steps.
Not every decision domain has a sufficient and clear information store like a git codebase. Therefore, collecting the information needed from human feedback to build the right information store is a more natural way.
There’s a saying keeps coming back to my ear that the most valuable asset in the agent era is the user information. But the key question is why and how do we collect it?
A recursive, auto-speciation agents system might be the answer. There are only two things that matter in this vision:
First, build the recursive agents loop system that actually solves problems.
Second, build the user-friendly decision-in experience.
Ending
AI may not act like Jarvis in Iron Man, or Joi in Blade Runner 2049, given Transformer-based LLMs. But it has a good chance to be able to autonomously solve super complex problems while humans only have to give it a few incentives.