Harness²: Recursive Agent Harnessing for an Open World

Frozen Weights, Editable Everything Around Them

Almost every attempt to make agents better over the past two years has happened in the weights: bigger models, longer post-training, more tool-use traces. But an agent in production is never only its weights. Take the same model and wrap it in a well-written system prompt, a couple of well-scoped skills, and a hook that catches malformed tool calls, and its completion rate can differ by orders of magnitude from the same model behind a bare prompt. Google Research calls that surrounding layer a harness: the system around a model that lets it act as an agent — prompts, tools, memory, skills, orchestration. Harnessing is the work of optimizing that system.

The trouble is that harnessing has been a craft, not an experiment. An engineer reads a failed trajectory, adds a paragraph to the system prompt on intuition, reruns, and eyeballs whether things look better. There is no control condition, no accumulating evidence, and no way to answer which part of the change actually did the work. The open-world premise makes it worse: the tasks an agent meets were never seen when the harness was designed, so a prompt tuned for one domain can easily be net-negative in the next.

Harness² (read as "harness squared") is the framework Google Research released in October 2026 for exactly this, with the paper at arXiv:2610.14224, Harness²: Recursive Agent Harnessing for an Open World. In one sentence: it turns tuning the harness into a recursive, measurable, accumulating experiment while the model weights stay completely frozen. The project lives at harness2.github.io under Apache-2.0, and the repository states plainly that it is not an officially supported Google product.

Two Recursions, Neither Touching Parameters

The framework is two recursions locked together.

Task-level recursion refines the execution harness for the task in front of it through a contrastive Propose–Probe–Compose step. An improver proposes candidate edits; each candidate is actually executed on the same task (that is the probe); and a composer then looks at the behavioral contrasts between those executions to synthesize a refined harness. The important half is the last one — there is no ground-truth feedback anywhere in the loop. It never asks what score this run got. It asks what the agent did differently at step three once a rule was present. That is what makes the method usable in an open world, where real tasks usually have no queryable grader.

Harnessing-level recursion widens the horizon to a stream of tasks. After each task, a persistent improvement harness is updated with the harness that was adopted and with lessons distilled from the execution evidence that the evidence actually supports. The improver for the next task reads that growing record.

So the first level asks what harness this task needs; the second asks how to change harnesses so that changes are more often right. Both consume execution trajectories only, and the weights never move. That is the sharpest line between this and RL-based agent tuning: the object being optimized is the shape of context and tools, not parameters.

Eight Editable Components, and a Counter-Intuitive Cost Ordering

Harness² declares the editable surface as eight components: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and plugins (numbered c1 through c8 in the source). Between them they control context construction, the tool interface, delegation, and behavior inside the tool loop.

The interesting part is the cost ordering the framework hands to the improver, because it runs against intuition. On the OpenCode surface, the eight components map to paths with very different lifetimes in context:

systemprompt.md                     c1 role and procedure   ALWAYS ON -- every token competes with the output budget
guardrails.md                       c6 standing rules       ALWAYS ON -- same cost
LongTermMEMORY.md                   c3 durable domain facts ALWAYS ON -- same cost
tool_descriptions/tool_guidance.md  c4 tool prose           ALWAYS ON -- same cost
skills/<name>/SKILL.md              c6 reusable workflow    ON DEMAND, loaded only when invoked
sub_agents/<name>/AGENT.md          c5 delegated subagent   ON DEMAND, own context, returns text
scripts/<name>.py                   c7 deterministic helper ON DEMAND, run through Bash
plugin/<name>.ts                    c8 plugin               always registered, acts INSIDE the tool loop

The prompt text is blunt: always-on prose is the expensive option. Every word of it is in context on every run, competing with the agent's own output budget, and the costliest failure on this kind of work is a run that reasons until the budget is gone and delivers nothing. Adding prose makes that failure more likely, not less. An on-demand component costs nothing until the moment it is invoked.

Hence the stated preference order: a skill for anything with a recognizable trigger, which is the default form for new instruction; a script for a deterministic operation the runs hand-rolled and got wrong or did slowly; a subagent for a bounded investigation whose input is large and whose answer is small (read twelve PDFs, return one normalized table — a bad trade the moment the output is as big as the input); and a plugin only when a recurring mechanical problem in the tool loop is better handled there than in prose, because prose can be stopped reading and a hook cannot.

The two surfaces also differ in hard caps. OpenCode allows 24,000 characters of always-on text, with the reason recorded in code: every token there competes with the agent's own output budget. Codex has a single always-on file, AGENTS.md, capped at 8,000 characters, with a sharper comment: it is injected whole into every turn, and Codex additionally truncates it silently at 32,768 bytes. The same improvement algorithm therefore sees different writable paths, naming rules, frontmatter requirements, and trigger conventions on each surface — and all of that difference is collapsed into data on a SurfaceSpec, so the shared logic exists once.

The edit contract is omitted-is-inherited: a proposal emits only what it changes, and everything else carries over. Proposals then go through full validation. Skill frontmatter needs a name: matching its directory and a description:, or it is dropped silently at load; a skill must be named in the always-on text, and if it is not, autotrigger() repairs it by appending a trigger line to guardrails.md (skipping the repair when the line would push the body over the always-on cap); any script a skill or always-on file invokes must ship with it; plugins and hooks files are schema-validated, because a malformed hook never fires rather than erroring. An invalid proposal gets one retry, and if the retry fails the stage inherits the current harness and records why.

Evidence Ships as Files, Never Truncated to Fit a Prompt

The improver does not receive a summary. It receives a full evidence tree on disk, laid out as core/context.py documents it:

context/task.md                  the task as the agent received it
context/rollouts/<label>.md      full trajectory -- no prompt-budget truncation
context/deliverables/<label>.md  the artifact, via the judge's own extractor
context/harness/<rel>            the c1..c8 tree being edited
context/library/INDEX.md         earlier tasks in this domain + their full harnesses
context/pool/<label>/            candidate harness and its rollout (compose / select only)
MISSION.md                       what to do with all of it

The design stance is one sentence: evidence is served as files and the prompt is a short pointer to them, so nothing is truncated to fit a prompt. A single observation may be clipped, but a trajectory as a whole is written out in full.

Paired with that is a set of anti-leak boundaries, which is what lets the method double as an evaluation. assert_no_leak_paths() rejects benchmark-declared answer-key paths before any context reaches a model. assert_no_grader_verdicts() refuses a context tree containing the grader's own output, checked by file name and by content. assert_tree_has_no_grader_verdicts() applies the same check to an agent-authored harness tree before it can be rolled or stored, because such a tree could never afterwards be served as evidence. The code comment is careful about what this check is: a tripwire on the adapters' read boundary, not the defense itself — if it fires, find the read path that produced the match rather than widening the marker list.

The benchmark adapters do one more thing on top: they strip the grading criteria. The LAB adapter groups tasks by practice area and removes criteria from the material passed to the improver; JobBench keeps its rubrics outside the improver's evidence; the WorkBuddy adapter reads the verifier's score.json while the improver never receives verifier output.

The Budget Identity: R = 4k Rollouts, P = 3k Improvement Stages

The cost of a run is exactly computable, and core/ledger.py is the account book:

R = 4k rollouts          = k shared base executions + 2k candidate probes + k finals
P = 3k improvement stages = k invariant proposals + k domain proposals + k compositions
                            (agg1..aggk in parallel mode, c1..ck in sequential mode)

Base executions count toward each mode, retries are counted as separate invocations, and selection and reflection are recorded outside this ledger. If the account does not land on the identity, a BudgetError is raised. That is not bookkeeping for its own sake; it makes "what does one more unit of recursion budget actually buy" a question with a precise answer.

Selection is blind. The pool is exactly the k finals that the mode produced — reference executions are not in it, so abstention does not exist by construction — and each output is anonymized to A, B, C and so on. The selector sees only anonymized deliverables, picks one, and only then reflects on that candidate's refinement evidence. At k=1 no selector session runs at all; the single final is delivered directly, but still written with the standard selection.json schema at the same path so judge watchers find it once. A selector failure falls back to a delivered base and never raises; an adapter bug does raise, because that is a bug and not a selection outcome.

Judging happens after selection, and core/judge.py is emphatic about its place: nothing in the method imports this module. A judge failure is None, never a zero, and a None is excluded from the denominator. Reporting follows the same rule — ungraded rows and tasks that never completed are counted and then excluded rather than scored as failures. That is what keeps a headline number from being quietly dragged down by "crashed runs count as zero."

Parallel Explores Wider, Sequential Refines Deeper

Task-level recursion scales two ways, and both start from k reference executions of the same base harness.

Parallel recursion (--mode parallel, stored on disk as agg) proposes and probes k general (invariant) candidates plus k domain-specific ones, then runs k independent compositions, each of which sees the full candidate pool and all execution evidence. Sequential recursion (--mode sequential, stored as seq) runs k Propose–Probe–Compose steps, carrying the refined harness and the accumulated evidence into the next step.

The README compresses the trade-off into one line: parallel recursion explores wider, sequential recursion refines deeper. On the command line the knobs are --k for the task-level budget and --mode for the shape, plus --workers N to run N domains concurrently while tasks inside a domain stay ordered, --substrate cx to switch to the Codex executor, and --seed N to shuffle the task stream reproducibly without controlling model sampling.

One Library per Domain, Carrying Whole Trees Rather Than Summaries

Harnessing-level recursion lands in a per-domain experience library R, implemented in core/library.py with a fixed on-disk shape:

<root>/<run-tag>/<mode>/<domain>/INDEX.md
<root>/<run-tag>/<mode>/<domain>/harnesses/<task>/harness/<rel>   the full adopted tree
<root>/<run-tag>/<mode>/<domain>/harnesses/<task>/EXPERIENCE.md   why picked + lesson

Three details carry weight. First, retrieval is by construction: a domain's directory only ever holds same-domain entries, so there is no vector search and no relevance scoring — the path is the filter. Second, the whole tree is carried, never a summary, an explicit choice in the module docstring that spends disk in exchange for letting a later improver read every word the previous task actually adopted. Third, tasks within a domain run in stream order and read only earlier entries, while different domains may run concurrently; libraries are separated by benchmark, executor, mode, budget, and seed, so every sweep starts every domain empty.

What counts as a lesson is also gated: has_experience() requires cue, lesson, and evidence to all be non-empty strings, so an unsupported reflection does not enter the library. That matches the paper's phrasing about evidence-supported lessons.

Three Benchmarks, Two Executors

The release wires up three professional-work benchmarks spanning legal, general professional, and office/web/code work:

Benchmark	Upstream	Task shape	Install and extra deps	Grading
LAB	harveyai/harvey-labs	Legal work with source documents and rubric-based grading; the full stream is 1,143 tasks, grouped by the adapter into practice areas	harness2-setup lab; needs Pandoc and Poppler	Partial (fraction of criteria passed) and All-pass (every criterion passed)
JobBench	Job-Bench/job-bench-eval; data at HuggingFace JobBench/job-bench	Professional tasks, with the library grouped by the dataset's occupation directories	harness2-setup jb; the dataset is public and ungated, and setup records the HuggingFace revision it downloaded	Upstream rubric weights; unscored tasks are reported separately
WorkBuddy-Bench	Tencent/workbuddy-bench	office, web, and code subsets; each rollout gets a fresh benchmark container	harness2-setup wb --subset office; needs Docker, and the upstream Tencent custom license carries a territorial limitation you should read first	Adapter reads the verifier's score.json; missing results stay unscored

There are two substrates: OpenCode by default, and Codex behind --substrate cx. Both implement the single interface in substrate/base.py, and benchmark adapters supply only the launcher body. Containerized benchmarks need one extra piece for Codex — harness2-bridge --host 0.0.0.0 serves a Responses-API endpoint on Vertex AI so the containers can reach it.

Models are configured in .env with four settings that matter: HARNESS2_SOLVER_MODEL (executor model in OpenCode's provider/model format), HARNESS2_IMPROVER_MODEL (defaults to the executor), HARNESS2_JUDGE_MODEL (a bare model ID; the adapter supplies the Vertex prefix), and HARNESS2_RUNS_DIR (output root, default runs/). Shell variables win over .env. One piece of defensive design is worth copying: resolve_channel_model() requires channel and model to be overridden together and raises otherwise, because a bare ID against a provider-prefixed channel fails silently downstream.

What It Measured

The headline result spans three professional benchmarks and four model–harness configurations: a single Harness² step improves GLM-5 and Gemini 3.5 Flash over their base harnesses, by up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB's partial pass rate. Raising the task-level recursion budget brings further gains.

Two caveats matter when reading those numbers. They are for a single step, the --k 1 setting, which costs exactly four rollouts and three improvement stages — a cheap intervention for that much movement. And they are stated for two specific models against their own base harnesses, not as a universal gain across models.

For baseline comparison, harness2-judge --bases scores the reference executions in a run namespace and writes base_judged.json beside judged.json, while harness2-report covers the refined harness only. The two are not interchangeable.

Running It

The environment is Linux, Python 3.12, and uv (the package also supports 3.13). The local demo needs no model credentials at all:

uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

That demo runs both recursion modes with deterministic editors and real Python executions — the orchestration is genuine, and only the improver's judgment is replaced by fixed rules. It is the cheapest way to understand how the pieces fit.

Model-backed runs additionally need Node.js 20+, the Google Cloud CLI, and each benchmark's own dependencies; then cp .env.example .env, fill in the Vertex AI project and model IDs, and run gcloud auth application-default login. A full benchmark pass is five commands:

harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
  --only-domains antitrust-competition --max-tasks 1
harness2-judge  --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

Refinement, judging, and reporting must agree on benchmark, executor, subset, --k, --mode, and seed. --only-domains restricts the run to named domains, --max-tasks caps tasks per domain rather than across the run, and --dry-run previews task selection with no model calls. Before a real sweep, harness2 checks the judge offline first, because the namespace records the judge model and harness2-judge refuses to grade it under a different one — discovering a missing judge after the sweep means paying for the sweep twice.

Block the Network First: a Hard Boundary on Validity

This is the most quotable piece of self-disclosure in the repository. All three benchmarks are public, and JobBench publishes each task's RUBRICS.json on HuggingFace. An executor with network access can therefore retrieve the grading criteria — and in a validation run of this release, a Codex rollout on JobBench downloaded its task's rubric without being instructed to.

The release does not restrict network access, and Codex runs with its sandbox disabled. So the instruction is to block, at the network level, huggingface.co, hf.co, *.hf.co, plus github.com and raw.githubusercontent.com for benchmark repositories, and to report the restrictions used alongside the results. Setup itself needs those hosts, so the order cannot be reversed, and a tool-level setting is explicitly not enough — apply an egress firewall or DNS policy on the executor host.

Putting this in the README rather than an appendix is worth more than the finding itself. It amounts to conceding that a score on a public benchmark only means something if the evaluator did the isolation work; otherwise what is being measured may be the agent's ability to find the answer key.

Reproducibility and Resume

Run namespaces are built in exactly one place, ns() in core/schedule.py, as <bench>-<substrate>-<mode><k>[-s<seed>][-<tag>], with output under runs/lab/, runs/jb/, and runs/wb/<subset>/ holding task summaries, execution artifacts, adopted harnesses, reflection notes, and graded results. Changing models or task selection requires a new HARNESS2_RUNS_DIR; the saved configuration is what prevents an incompatible resume.

Resume does not key off "is there output in the directory." core/runmark.py records completion explicitly, and only the code that observed the runner return normally may write the marker. The marker also fingerprints the harness tree that produced the rollout, so a rollout from a different configuration is never reused, and byproducts that are not part of the harness are excluded from that fingerprint. Rerunning the same command resumes.

On concurrency: tasks within a domain stay strictly ordered because the library depends on it, domains run in parallel under --workers, and summary files are merged atomically under an flock since the two modes may be written by separate processes. Task-stream order is constructed in one place by stream_order() — seed=None gives the reference order (domains by descending task count then name, tasks sorted within a domain), and an integer seed shuffles hierarchically with Random(f"{seed}:__domains__") and Random(f"{seed}:{domain}"). For a reproducible domain-stratified partition, scripts/split_tasks.py --bench lab --seed 42 --output splits/lab.json writes a local split file (default train fraction 0.5, refuses to overwrite), which you then run with --split-file and --split test.

One reproducibility trap is called out explicitly: JobBench's dataset is resolved at download time rather than pinned to a commit, and setup records the downloaded HuggingFace revision in dependencies/job-bench-eval/dataset/main/.harness2-download.json — report that revision alongside any JobBench result. By contrast, DEPENDENCY_PINS in config.py pins job-bench-eval, workbuddy-bench, harvey-labs, and opencode to specific commits.

How to Read This Project

Harness² is not an agent framework and does not want to replace OpenCode or Codex; it treats both as swappable surfaces and sits one level above them. What it actually ships is infrastructure for turning harness tuning from craft into experiment: the editable surface is declared and validated rather than implied, evidence lands on disk in full instead of being squeezed into a prompt, the budget is an identity you can check, selection is blind, judging happens after selection, failures are unscored rather than zero, and experience accumulates per domain with only evidence-backed lessons admitted.

For teams building agent products, the most transferable parts are three engineering conclusions. Always-on prompt text is the most expensive lever, so anything that can be a skill, a script, or a subagent should not be prose. Every harness change deserves a control execution on the same task, or you cannot tell an improvement from luck. And a score on a public benchmark means nothing until the executor's network has been isolated.

The scale also marks this as a research release rather than a product: roughly 8,500 lines of Python under harness2/, with the three heaviest files being core/prompts.py (1,163 lines — every improver and selector prompt), core/schedule.py (850 lines — namespacing and the three drivers), and core/recursion.py (578 lines — both recursion modes), plus third_party/ holding the runner overlays, patches, and license notices for each benchmark.