From AutoGPT to Agent Harness: What Changed in Three Years?

The first time I seriously engaged with the idea of an "AI Agent" was in 2023, through Lilian Weng's now-classic post LLM Powered Autonomous Agents.

The definition I kept was the one Lilian distilled:

The LLM is the agent's brain. Around it, we add Planning so it can break a complex task into subtasks; Memory so it can retain information beyond the context window; and Tools so it can search the web, run code, call external APIs, and actually act on the outside world.

The Agent projects that went viral at the time were AutoGPT and BabyAGI.

Looking back, Agents in 2023 were a bit ahead of their time: fun as a concept, not very useful in practice.

When AutoGPT and BabyAGI exploded in the first half of 2023, OpenAI hadn't even shipped native function calling yet (that landed in June 2023; reliable JSON mode waited until DevDay in November). If you wanted a model to call tools, you had to prompt it into a specific response format, parse it yourself, and run the corresponding command.

OpenAI's first reasoning model, o1, was still more than a year away. In 2023, the main way people pushed complex reasoning was still prompting tricks like "think step-by-step."

Context windows and cost were also nowhere near today's scale. The original GPT-4 defaulted to 8K context; the 32K version was still limited access. GPT-4's input price was billed per 1K tokens, which works out to $60 per 1M tokens: six times more expensive than today's Claude Fable 5.

In other words, we were trying to turn a model that:

• had no native tool calling and no reliable structured output;
• was not a reasoning model;
• had a context window that couldn't even fit today's Claude Code system prompt + tool definitions

into an Agent that could work autonomously on long-horizon tasks.

Agents in 2023: AutoGPT

AutoGPT is a typical example.

It tried to let GPT-4 break down goals, call tools, and keep pushing a task to completion without continuous human intervention.

Its core structure looked roughly like this:

In that loop, the LLM wasn't just answering questions. It was asked to keep outputting the "next action." Those actions could be searching the web, reading and writing files, running shell commands... The system parsed the command, executed it, and fed the result back to the model as an observation.

Because OpenAI hadn't shipped native function calling yet, the loop was nowhere near as clean as today's. AutoGPT relied on prompt engineering to make the model emit commands in a agreed-upon format, then external code handled parsing, validation, and execution. Architecturally, though, it was already basically today's tool-use agent.

2023–2026: The Loop Got Simpler

Today, with any model API that supports native tool calling, you can implement a minimal Agent in a few dozen lines:

while (true) {
  const response = await model(messages, tools);
 
  if (!response.toolCalls.length) {
    return response;
  }
 
  const results = await executeTools(response.toolCalls);
 
  messages.push(response);
  messages.push(...results);
}

If the model wants to keep working, it calls a tool.

After the tool finishes, the result goes back into context.

When the model decides the task is done, it returns a final response.

Some architectures that looked essential in 2023 are no longer needed.

Early AutoGPT made the model explicitly output Thoughts, Reasoning, Plan, and Criticism. The idea was: if the model isn't good enough at long-horizon reasoning yet, the framework should break cognition into a few explicit steps for it.

But as models got stronger, we no longer needed a PlannerAgent, plus an ExecutorAgent, plus a CriticAgent. Agent frameworks no longer had to simulate thinking through ever more elaborate orchestration.

A modern coding agent can give the model a few very primitive tools: read, write, edit, execute. Then let the model decide which files to read, what to search, what to change, which tests to run, how to recover from failures, and when to stop.

Which leads to a counterintuitive pattern: as models get stronger, the Agent Loop gets simpler; but production-ready Agent systems get more complex.

Where did the complexity go?

Into the Harness around the loop.

Agent Harness in 2026

In 2026, "Agent Harness" suddenly became a popular term.

If the Agent Loop is: Model → Tool → Model → Tool → ...

then the Harness is the entire runtime environment wrapped around that loop:

There are at least five kinds of problems here.

1. Execution: How does the agent keep working?

This is the part closest to the 2023 Agent Loop: tools, tool execution, stop conditions, retries, orchestration, subagents...

2. Context: What does the model know right now?

This is more than dumping messages[] into the model. System instructions, project instructions, skills, the current task, recent tool outputs, relevant files, compacted history summaries... all of these compete for a limited context window.

The Harness has to decide: what enters context, what gets truncated, which history needs compaction, what loads dynamically, and whether the context policy should change when you switch models...

3. State: What survives across turns?

The simplest chatbot can treat a session as:

const messages = [];

But a long-running Agent needs conversation history, tool execution history, pending approvals, workspace, usage, compaction summaries, checkpoints, branches, and execution state.

So a real Agent session looks more like an execution log than a chat history.

4. Environment: Where does the agent actually work?

This is especially obvious for coding agents.

Giving the model an execute tool isn't enough. Where does that shell run? Is the filesystem persistent? How long can processes live? Can it reach the public internet? How are secrets injected? After a session resume, is the workspace still there? If the code gets broken, can you roll back?

Once an Agent starts acting on the outside world, tool calling quickly turns into a runtime and infrastructure problem.

5. Control: What is the agent allowed to do?

A common 2023 move was to tell the model in the prompt:

Be efficient. Do not do anything dangerous.

But a prompt is an instruction, not enforcement.

A more mature Harness puts actions behind a policy boundary:

Agent wants to execute action
            ↓
        Policy Engine
            ↓
    allow / deny / ask
            ↓
      Execute Action

The model can decide "what to do." The Harness decides whether that action is actually allowed to happen.

Put 2023 and 2026 side by side:

One clear shift: the responsibility boundary between Model and Harness was redrawn.

In 2023, models weren't capable enough, so we piled cognitive scaffolding into the framework: Planner, Critic, Task Creator, Prioritizer, Memory Retriever. We tried to tell the model how to think through orchestration.

Today, the model itself can take on more reasoning, planning, and tool selection. So frameworks can step back and use the harness to define the execution boundary.

The model owns decisions that essentially require intelligence:

What should I do next?
Which file should I read?
What should I search for?
What does this error mean?
Is the task done?

The Harness owns the things you shouldn't leave to the model's good intentions:

What context can you see right now?
Which tools can you access?
Can this command run?
Where does it run?
Which state must be persisted?
Does the user need to approve?
How does the system recover after a failure?

Looking at Agent SDKs through this lens

I've been building an Agent for myself. While picking a framework, I went through every name that comes up: LangChain, LangGraph, Vercel AI SDK, eve, OpenAI Agents SDK, Pi, Claude Agent SDK, Codex SDK...

Reading the docs, one thing was obvious: they all call themselves "Agent SDKs," but comparing them by feature list is pointless. They aren't even solving problems at the same layer. The distinguishing question is to take the five Harness decisions above one by one and ask: for Execution, Context, State, Environment, and Control, does each decision belong to you, or to the framework?

Take the LangChain ecosystem. It stacks three products (docs):

At the bottom, LangGraph gives you ready-made execution machinery: checkpointing, persistence, human-in-the-loop. But it has no opinion about what an Agent itself should look like. In LangGraph, an Agent is a graph you draw yourself: which node calls the model, which node runs deterministic code, which edge fires under which condition, where to pause for human approval. All of that is yours to define. LangGraph owns the engineering problems: if the process crashes, how does the task resume from the last checkpoint instead of starting over; if a session is halfway done, how does it still exist after a machine restart; if you're waiting hours for approval, how does the task suspend without burning compute. The Agent design questions: which tools to give the model, how to manage context, whether to split work to a subagent, when the task counts as done. It answers none of those.

The middle layer, LangChain, starts answering the first batch of design questions. create_agent() hands you a ready-made tool-calling loop, so you don't have to draw the graph yourself (under the hood it compiles into a LangGraph graph to run). Capabilities outside the loop attach as middleware: summarization, human-in-the-loop, model retry, tool-call limits, PII filtering all ship as optional pieces. In other words, "what the loop looks like" is already decided for you, but which tools the model gets and how the Agent actually does its work are still yours.

At the top, Deep Agents gives default answers to almost all of the remaining design questions. How to plan (built-in write_todos tool), how to manage context (virtual filesystem plus auto-summarization), how to decompose tasks (subagents), what the Agent is allowed to do (filesystem permissions and human-in-the-loop). The first line of the README is "The batteries-included agent harness": what you get out of the box is a complete Agent that can start working. Your job is mostly to override and replace its defaults.

So inside one ecosystem, engineering problems are owned by LangGraph, while design decisions get absorbed layer by layer from LangChain up to Deep Agents: the higher you go, the more decisions the framework makes for you, and the less room left for the developer.

Honestly, though, I really dislike the LangChain family. Every layer asks you to accept a preset conceptual system first: in LangGraph, an Agent must be expressed as State + Nodes + Edges; in LangChain, capabilities must be written as middleware and hung onto the loop. Developers have to buy into that mindset and work down through its abstractions, often fighting the framework. And the Agent space is changing so fast that today's abstractions are likely tomorrow's throwaways. Yage had a good take on this last year in Why the First Step in Learning Agentic AI Is to Forget All Frameworks: picking an opinionated framework today isn't picking a feature list. It's picking a worldview.

Among these options, the route I prefer is the Vercel AI SDK: it started by unifying model providers, streaming, and tool calling; in v5 and v6 it gradually absorbed agent loops, durability, and approvals; but it stayed a thin abstraction. Most of the five decisions stay in your hands. Whether to give them away, and when, is up to you. Vercel put the default harness-layer answers in eve, which just shipped last month (built on top of the AI SDK, the same way Deep Agents is built on LangGraph). eve's design idea is "An agent is a directory." Developers use a directory (instructions, tools, skills, subagents) to answer "who is this Agent, what does it know, what can it do," while eve mainly owns "how does it live reliably in production": durable execution, sandboxed compute, approval, and evals all come by default.

OpenAI Agents SDK owns runtime decisions for you: Runner handles turns and tools, with native sessions, handoffs, and guardrails. OpenAI's docs say outright that if you want to own the underlying loop, don't use Agents SDK; use the Responses API.

Pi is a complete coding-agent harness: session persistence, branching, compaction, and skills are all there, but it stays deliberately hackable. Almost every decision can be swapped out through an extension. Claude Agent SDK and Codex SDK are already tuned agents: nearly all five decisions belong to the vendor. What you still own is instructions, tools, and where it runs.

Put them in a table:

So choosing a framework is really about deciding: of these five kinds of decisions, which ones do I want to own, and which am I willing to hand over? If you want a long-horizon agent out of the box, use Deep Agents or Pi. If you just want to embed a battle-tested agent into a product, use Claude Agent SDK or Codex SDK. If you want to keep as many decisions as possible, start from a thin abstraction like the AI SDK.

While looking at the LangChain ecosystem, I noticed two things that happened to Deep Agents this year.

First, in February LangChain published Improving Deep Agents with harness engineering. They froze the model on gpt-5.2-codex and changed only the harness: system prompt, tools, middleware. Their coding agent went from 52.8 to 66.5 on Terminal Bench 2.0, from Top 30 into Top 5.

Second, at the end of July, Deep Agents v0.7 did a major simplification. The default system prompt was removed entirely, built-in tool descriptions were cut by 43%, and the planning middleware went from on-by-default to opt-in. Base input tokens on a default agent turn dropped from 5,395 to 1,895, a 65% reduction, with no quality regression on their evals.

Taken together: the Harness decides how much of a model's ability you actually get to use, so it's worth serious engineering. But what belongs inside the Harness depends on the model's capability boundary right now. Every time the model gets stronger, another batch of cognitive scaffolding can be deleted.

This post was originally written in Chinese and translated to English by Grok 4.5. The original version is here.