Skip to content
←Back to Papers

PAPER DEEP DIVE

Agent HarnessHarness DistillationSupervised Fine-Tuning

Harness-Zero: Harness Distillation via Agent-as-Harness

Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and available information, so guidance from the optimized harness cannot serve directly as supervision for the target one. We introduce Harness-Zero, which enables harness distillation through agent-as-harness. Guided by the optimized harness, a harnessing agent corrects student responses before execution in the target harness's action space, turning harness guidance into training demonstrations. Fine-tuning on the resulting trajectories internalizes harness-induced behavior into the model, so the specialized harness can be removed at deployment. Our experiments spanning knowledge work, tool use, and science domains show that: (1) For frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness. (2) With the specialized harness removed at deployment, Harness-Zero improves the base model's macro-average task success from 23.3% to 44.3%, even exceeding the 41.7% it reaches with that harness still attached. (3) Harness-Zero recovers harness-induced behaviors absent from the base model, with 82.3% average recovery across 28 patterns in the three domains.

Paper: Harness-Zero: Harness Distillation via Agent-as-Harness

Authors: Haoran Ye, Yuxing Lu, Haonan Dong, Zhaochen Su, Guojie Song

Affiliations: Peking University, Google, Hong Kong University of Science and Technology

Version: arXiv 2609.24974v1, September 21, 2026

Paper: https://arxiv.org/abs/2609.24974

Code: metaevo-ai/harness-zero, Apache-2.0. This analysis audited commit 8697d54462e45caaa239b0a999acacd36399435b.

One-sentence summary: Harness-Zero uses a private guide adapted from an optimized agent harness to review and correct every unexecuted student response, then performs supervised fine-tuning on the accepted trajectories so the optimized harness behavior survives inside the model when only a minimal harness is deployed.

Why Harness Capability Is Usually Trapped Outside the Model

A language-model agent is not defined by its weights alone. The surrounding harness determines which tools the model can call, how observations are formatted, which middleware can block an action, what skills and memory are available, and when the interaction loop terminates. The paper uses the term agent harness for this external system. Its influence is large enough that the same base model can exhibit very different success rates under different harnesses, making harness design part of the agent’s effective capability.

This creates a deployment problem. A harness optimized for spreadsheets may be poor for multi-application workflows. A harness tuned around one model may become brittle after the model is updated. An agent intended to handle many domains can either use one compromise harness or route among an expanding set of specialized systems. The first choice caps performance; the second raises deployment complexity, maintenance cost, and context-switching risk.

Prior work follows two broad strategies. Code-as-harness optimizes prompts, tools, and middleware directly, and then deploys that optimized code around the model. Model training improves the policy inside a chosen system but does not explicitly transfer behavior discovered through harness optimization. Harness-Zero asks whether the gain of an optimized harness can become training signal rather than remaining a permanent deployment dependency.

The problem is not ordinary imitation learning. A trajectory collected under the optimized harness may contain tools absent from the target harness, or observations that the student is not allowed to see at deployment. Direct cloning can therefore teach commands that cannot execute or create dependence on information that will disappear. The technical task is to convert harness guidance across action spaces while preserving the procedure that made the source harness effective.

The authors call this problem agent harness distillation. The optimized harness is available during training, while a fixed target harness remains at deployment. The resulting behavior is stored in model parameters. This framing connects two normally separate forms of agent engineering: improving the external scaffold and improving the internal policy. It also states the scope precisely. Harness-Zero removes a specialized harness, not the need for every harness.

Notation: Three Harnesses with Different Jobs

The target harness h is the fixed deployment system. In the experiments it is a minimal mini-SWE-agent-style harness with a fixed system prompt and one Bash execution tool. The optimized student-side harness h* is evolved on training tasks and may contain additional tools, middleware, skills, and memory. The private reference harness K is adapted from h* so that a reviewing agent can use it. The three objects are related, but they are not interchangeable.

Let πθh denote the response policy of a model with parameters θ while operating under h. Let Th(c,y) denote the transition that executes response y from context c through h and returns the next student-visible context. This notation separates generated text from environmental execution. It also makes action-space mismatch explicit: a model can emit a string, but only the harness can decide whether that string is executable and how its result appears in the next context.

Agent-as-harness does not replace the sandbox with another free-form agent. It inserts a private review boundary around each student response. The student keeps acting under h, and h still executes the accepted action. The reviewer reads guidance adapted from h* and decides whether to pass or replace the proposal. The source harness can therefore shape the rollout without injecting inaccessible tools or hidden observations into the deployed student context.

The Three-Stage Method

The first stage evolves h* on training tasks and adapts it into K relative to h. The second stage collects trajectories by letting the student propose responses under h and asking the reviewer to intercept every response before execution. The third stage filters successful trajectories and fine-tunes the student on the accepted responses. At deployment, h*, K, and the reviewer are removed. Only the distilled policy and h remain.

flowchart LR
    A["Training tasks D_train"] --> B["Evolve optimized harness h*"]
    B --> C["Adapt h* against target h"]
    C --> D["Private review guide K"]
    E["Student proposal y_t under h"] --> F{"Review before execution"}
    D --> F
    F -->|PASS| G["Accept original y_t"]
    F -->|REPLACE| H["Build valid z_t under h"]
    G --> I["Execute only accepted response"]
    H --> I
    I --> J["Return student-visible observation c_t+1"]
    J --> E
    I --> K["Keep successful trajectories"]
    K --> L["SFT on accepted responses"]
    L --> M["Deploy student with h only"]
    

Figure 1 presents the same three stages. The left panel evolves the expert harness and translates its components into a private reference. The center panel shows one student rollout wrapped by a reviewer that can pass or replace each proposal. The right panel masks reviewer-perspective reasoning, trains on retained trajectories, and deploys without the training-time systems. The examples at the bottom show that review is not synonymous with constant intervention. A student that has already inspected a workbook can receive PASS, while a premature completion can be replaced by a response that continues the required file operations.

Overview of Harness-Zero

Figure 1: The Harness-Zero pipeline for harness evolution, reviewed rollout collection, SFT, and minimal-harness deployment. Image source: paper Figure 1.

Stage One: Evolve a Harness, Then Adapt It for Review

Given a training set Dtrain, the first stage performs two operations:

$$ h^{\star}=\operatorname{Evolve}(\mathcal{D}_{\mathrm{train}}), \qquad \mathcal{K}=\operatorname{Adapt}(h^{\star};h). $$

The first equation produces the optimized student-side harness. The second converts it into a guide for the reviewer. Adaptation depends on h because every correction ultimately has to be expressed and executed in the target action space. By keeping the two operations separate, the method avoids assuming that the expert harness and deployment harness share tools or observations.

The implementation uses a skill-guided evolution loop that analyzes recurring failures and updates a domain harness in three rounds. The resulting h* follows the DeepAgents abstraction, with tools, middleware, skills, and memory. This optimizer is not the main contribution and can be replaced by another automated harness-improvement method. Harness-Zero begins from the optimized system and studies how its induced behavior can become training data.

Adaptation changes both the audience and the enforcement point of each component. A source tool becomes a specification for constructing a student-native equivalent. Source middleware becomes review middleware that privately alerts the reviewer when a relevant condition appears. Skills and memory become diagnostic criteria and intervention guidance. A domain prompt may also tell the reviewer which failure patterns deserve particular attention.

These transformations preserve the intent of the optimized harness while producing responses that h can execute. The reviewer’s information access is intentionally narrow. It can read K, but it cannot inspect hidden solutions, verifier feedback, or environmental state beyond the student-visible context. If the reviewer needs evidence, it must propose an action available under h, such as a validation command, and let the result enter the student-visible trajectory. Corrections are therefore grounded in observations that the student can later reproduce.

Stage Two: Review at the Response Boundary

Code-as-harness runs the base model directly under the evolved harness:

$$ y_t\sim\pi_{\theta_0}^{h^{\star}}(\cdot\mid c_t), \qquad c_{t+1}=T_{h^{\star}}(c_t,y_t). $$

Both generation and execution are shaped by h*. The benefit remains tied to that system at deployment. Agent-as-harness instead starts with a proposal from the student under the target harness:

$$ y_t\sim\pi_{\theta_0}^{h}(\cdot\mid c_t). $$

The reviewer receives the visible context, the unexecuted proposal, the private guide, and its own history of earlier review decisions:

$$ (d_t,z_t)\sim\pi_H(\cdot\mid c_t,y_t,\mathcal{K},r_{{<t}}). $$

Here dt is PASS or REPLACE, zt is a complete replacement when one is needed, and r<t contains earlier private review exchanges and relevant reads from K. The accepted response is selected as follows:

$$ \tilde y_t= \begin{cases} y_t,&d_t=\textsc{pass}\\ z_t,&d_t=\textsc{replace} \end{cases} $$

Only the accepted response reaches the environment:

$$ c_{t+1}=T_h(c_t,\tilde y_t). $$

This distinction is central. Review is not a side conversation that later becomes extra context. It happens immediately before execution, and only the accepted response participates in the student-visible trajectory. The rejected proposal and the reason for rejection remain private. The resulting training sequence therefore has the same call-and-observation structure that the distilled model will encounter at deployment.

The reviewer is instructed to change as little as possible. It passes sound proposals unchanged and, when intervention is necessary, produces the smallest coherent full response that follows K. Full-response replacement matches the granularity of an agent turn: a turn can contain visible text, reasoning, and a tool call that must remain mutually consistent. The cost is coarse control, which the authors identify as a limitation and a target for future token-level or latent interventions.

One reviewing session spans the entire rollout. At each turn, the reviewer sees the student-visible events added since the previous review and the current proposal. This gives the correction enough local history to depend on state while avoiding a fresh replay of the whole environment at every step. A command can pass in one state and need replacement in another because the guide is interpreted against the current trajectory rather than compiled into static rules.

What the Repository Reveals About PASS and REPLACE

The released implementation makes the review boundary a typed middleware rather than a loose prompting convention. In src/harness_zero/student.py:98-208, HarnessReviewMiddleware wraps the model call, serializes the candidate response, validates its identifier, enforces turn and replacement budgets, and selects either the original message or a replacement. The decisive branch is compact:

if submission.decision == "PASS":
    accepted = candidate.original
    accepted_message = original_message
else:
    accepted = submission.replacement
    assert accepted is not None
    accepted_message = accepted.to_message(candidate_id=candidate_id)
    self._replacement_count += 1

self._store.append_review(candidate, submission, accepted)

Source: src/harness_zero/student.py:154-171. The environment receives a reconstructed accepted message, not arbitrary text returned by the reviewer.

The schema in src/harness_zero/review.py:15-96 also prevents ambiguous decisions. PASS cannot carry a replacement, REPLACE must include one, and a replacement must contain either visible content or a tool call. One submission is bound to one candidate. These constraints rule out partial patches, empty actions, and responses that claim to replace a candidate without supplying an executable turn.

if self.decision == "PASS" and self.replacement is not None:
    raise ValueError("PASS must not include a replacement")
if self.decision == "REPLACE" and self.replacement is None:
    raise ValueError("REPLACE requires a complete replacement")
if self.replacement is not None and not self.replacement.is_actionable:
    raise ValueError("a replacement must contain visible content or a tool call")

Source: src/harness_zero/review.py:65-72. A replacement must be an actionable response, not merely an instruction to try something else.

Stage Three: Learn the Procedure Without Learning the Reviewer’s Role

After collection, the accepted responses become supervised targets. For retained trajectories Dreview, the objective is:

$$ \hat\theta= \arg\min_\theta-\sum_{\tau\in\mathcal{D}_{\mathrm{review}}} \sum_{t=1}^{T_\tau} \log\pi_\theta^h(\tilde y_t\mid c_t). $$

The model is not trained to emit h* actions and is not given the reviewer’s private context. It learns to produce accepted responses from student-visible ct under h. The labels include both proposals that passed unchanged and proposals that were replaced. This combination preserves behavior the student already performs well while inserting optimized procedures where the student alone would miss them.

A subtle failure mode is reviewer leakage. A replacement can contain reasoning such as “this candidate is incomplete” or “I should replace the student’s proposal.” Those statements make sense only from the review role. If copied into an SFT target, the distilled student may comment on a nonexistent candidate at deployment or act as if it has private review authority. The data pipeline addresses this in src/harness_zero/sft.py:12-77 by retaining only successful, exception-free trials and identifying replacement turns whose reasoning contains review-perspective language. The training code masks those reasoning tokens from the loss.

if reward < reward_threshold or exception is not None:
    skipped_trials += 1
    continue

events = [
    json.loads(line)
    for line in events_path.read_text(encoding="utf-8").splitlines()
    if line.strip()
]

Source: src/harness_zero/sft.py:20-29. Failed or exceptional trials never become positive demonstrations.

src/harness_zero/dataset_utils.py:15-113 goes further by asserting that each serialized conversation contains only student-side messages. Private component paths, candidate files, the review submission tool, and review middleware names are rejected before training. This turns the information boundary into a data-construction invariant. Privacy is enforced by removing teacher-side structure from the sample, not by hoping that fine-tuning will cause the model to forget it.

The training recipe is sized for multi-turn tool use. Qwen3.5-9B is fine-tuned with LoRA rank 32, peak learning rate 2×10-4, final learning rate 1×10-6, batch size 8, maximum sequence length 65,536, and two epochs. The retained data contain 487 trials for SpreadsheetBench, 282 for AppWorld, and 500 for USPTO. The long context limit matters because accepted responses can include commands, file contents, and execution feedback rather than short question-answer pairs.

Deployment then follows a deliberately simple transition:

$$ y_t\sim\pi_{\hat\theta}^{h}(\cdot\mid c_t), \qquad c_{t+1}=T_h(c_t,y_t). $$

The target harness is unchanged; only the policy parameters differ. The model has internalized procedures that previously required the combination of an optimized harness and a reviewer. The deployed tuple is (πθhat, h), with h*, K, and the reviewer absent.

Experimental Design: Procedures, Interactive Tools, and Domain Knowledge

SpreadsheetBench Verified contains 400 real spreadsheet tasks. The authors use 300 for harness evolution and trajectory collection and hold out 100 for evaluation. The domain stresses state inspection, local edits, formula manipulation, and verification of the delivered file. Because all actions pass through a Bash tool, the harness must turn an underspecified request into a sequence of executable file operations.

AppWorld merges official train and development tasks into a 147-task training split and evaluates on 168 test_normal tasks grouped into 56 three-task scenarios. It requires interactive tool use across simulated applications. The metric is scenario goal completion rather than syntactic correctness of one API call. Success therefore depends on maintaining state and moving information across application boundaries.

USPTO Retrosynthesis uses 500 training tasks and 100 disjoint test tasks balanced over ten reaction classes. The model must predict single-step precursors. This setting combines executable procedures with chemical priors, candidate generation, and SMILES validation. It provides a useful contrast: a procedure can be demonstrated in a response, while a deeper knowledge prior may require much broader training coverage.

The inference comparison uses GPT-5.6 Sol and DeepSeek-V4-Pro. The distillation experiments use Qwen3.5-9B as the student and GPT-5.6 Sol as the reviewer. Harness evolution uses Kimi K3. Reasoning is enabled, with high effort when supported. In the training-free comparison, the same model can act as both student and reviewer, so the observed gain cannot be explained solely by inviting a stronger model to supervise a weaker one.

Result One: Extra Review Is Nearly Useless Without the Adapted Guide

Table 1 compares four conditions: the minimal harness h, the evolved harness h*, an agent reviewer with an empty guide, and an agent reviewer with K. Their averages are 68.6, 78.1, 69.2, and 81.1. Empty review improves the minimal harness by only 0.6 points. Another model call and a generic request to check the proposal do not create a reliable gain.

ConditionAverage successChange over hInterpretation
mini-SWE-agent under h68.6BaselineOne Bash action interface
meta-harness under h*78.1+9.5Use the evolved harness directly
Agent-as-harness, empty K69.2+0.6Review exists, but no expert guidance
Agent-as-harness, evolved K81.1+12.5Private guidance turns review into targeted correction

The averages aggregate six benchmark-model cells in the paper’s Table 1. Agent-as-harness with K is at least as good as h in every cell and improves the average over h* by 3.0 points. The pattern is not a uniform mechanical lift. On DeepSeek-V4-Pro AppWorld, h* scores 91.1 and agent-as-harness scores 89.3. The same model gains from 77.0 to 86.0 on SpreadsheetBench and from 56.0 to 60.0 on USPTO. The reviewer interprets the guide in context rather than reproducing one fixed harness trajectory.

The comparison also suggests a model-update advantage. A code harness encodes assumptions about how a model should act, and those assumptions can age when the base model changes. Agent-as-harness keeps that adaptation in the reviewing model, which reads the guide against the current trajectory. A more capable model may therefore apply the same concept more effectively. The advantage is conditional: the experiments later show that weak reviewers can make harmful decisions.

Table 1 from Harness-Zero

Figure 2: Paper Table 1, comparing code-as-harness and agent-as-harness before any model training. The public copy is scaled to the site image-width policy.

Result Two: Distillation Beats Keeping the Expert Harness

Table 2 provides the main evidence for harness distillation. Qwen3.5-9B under h succeeds on 23.3% of tasks on average. Mounting h* raises the result to 41.7%. After Harness-Zero fine-tuning, the same base model under h alone reaches 44.3%. The distillation produces a 21.0-point absolute gain and a 90.1% relative gain over the untrained student, and it slightly exceeds the result of retaining the expert harness.

SettingSpreadsheetBenchAppWorldUSPTOAverage
Base model under h31.026.812.023.3
Base model under h*39.048.238.041.7
Base model with DeepAgents35.019.67.020.5
Base model with Claude Code31.010.76.015.9
Harness-Zero under h44.058.930.044.3

The domain breakdown exposes different transfer dynamics. On SpreadsheetBench, the distilled model reaches 44.0 versus 39.0 with h*. On AppWorld it reaches 58.9 versus 48.2. These improvements fit the method: inspecting state, making a local change, verifying the result, and carrying information across tools are procedures that can be demonstrated in student-native turns and learned by SFT.

USPTO behaves differently. The distilled model improves from 12.0 to 30.0 but remains below h* at 38.0. The expert harness supplies reaction priors, candidate-generation logic, and executable SMILES validation, not only an action order. Response-level demonstrations expose part of that knowledge, but the student does not absorb all of it from the collected trajectories. The paper therefore separates procedural behavior, which transfers well, from deep domain knowledge, which may require broader pretraining, mid-training, or more data.

Generic harnesses do not provide a shortcut. DeepAgents averages 20.5 and Claude Code averages 15.9, both below the minimal h. The 9B student struggles with larger tool suites and longer contexts, while AppWorld and USPTO need constraints that broad tool collections do not supply. More available tools can increase distraction when the model lacks a domain procedure for using them. This result supports the paper’s distinction between adding a general scaffold and distilling a successful domain-specific one.

Table 2 from Harness-Zero

Figure 3: Paper Table 2, comparing the base policy, expert and generic harnesses, and the distilled policy across three domains.

Result Three: Supervision Must Match Both the Action Space and the Desired Behavior

The USPTO ablation is designed to separate “more successful data” from “the right kind of successful data.” Teacher rollouts succeed during collection 52.0% of the time but produce a distilled pass@1 of only 12.0%. A teacher operating under h* reaches a 62.0% collection rate but distills to 3.0%. The student running under h* collects at 39.4% and still tests at 12.0%.

Trajectory sourceCollection successDistilled test pass@1Change from base
Base model, no training—12.00.0
Teacher rollout52.012.00.0
Teacher under h*62.03.0-9.0
Student under h*39.412.00.0
Review with empty K44.211.0-1.0
Oracle answer98.615.0+3.0
Harness-Zero with K59.430.0+18.0

The source with the highest collection rate performs much worse than Harness-Zero. Oracle answers almost always complete the task, but they teach the result rather than a procedure the student can execute under h. Teacher rollouts may demonstrate successful outcomes without exposing the model-specific path needed by the student. Teacher actions under h* can be even less compatible with the target action space. Harness-Zero does not maximize collection success; it maximizes the usefulness of each accepted turn as a target-harness demonstration.

The 30.0 result against alternatives from 3.0 to 15.0 is stronger evidence than the aggregate score alone. It shows that a successful trajectory is not automatically good supervision. The response must be executable by the student, grounded in visible observations, and shaped by the behavior that harness optimization discovered. This is why the private guide and the target-harness boundary are both necessary.

Table 3 from Harness-Zero

Figure 4: Paper Table 3, comparing alternative supervision sources by collection success and post-distillation USPTO performance.

Behavior Recovery Shows What the Weights Actually Learned

The authors also measure whether the distilled model recovers specific behaviors that the expert harness induces and the base student lacks. They retain 28 patterns: 18 in SpreadsheetBench, 6 in USPTO, and 4 in AppWorld. For each pattern, recovery is the fraction of support tasks on which Harness-Zero exhibits the behavior. The average recovery rate is 82.3%.

The construction makes the comparison meaningful. By definition, the untrained student under h exhibits each pattern on 0% of its support tasks, while the same student under h* exhibits it on 100%. Harness-Zero is evaluated under h alone. Its 82.3% recovery therefore measures behavioral transfer rather than final-answer memorization. The patterns include inspecting inputs, avoiding premature completion, verifying edits, and using domain-specific validation procedures.

The behavior analysis also clarifies what remains outside the method. Some harness mechanisms, such as context compaction or internal context switching, are not naturally expressible as a student response. A reviewer can describe when such a mechanism is needed, but a full-response replacement cannot directly carry the implementation. Harness-Zero narrows the functionality that a deployment harness must supply; it does not eliminate every runtime capability.

Discussion: The Main Shift Is in Where Training Signal Comes From

Harness-Zero turns harness optimization into a temporary data generator and reviewer. The optimized system remains an external object during collection, but its effect is converted into parameter updates. This is different from treating runtime engineering and model training as separate tracks. It makes the artifacts of runtime search reusable as policy supervision.

The idea is relevant to embodied agents because robot stacks also place perception filters, skill libraries, recovery logic, safety checks, and workflow code outside the foundation model. Maintaining a domain-specific stack for every task is expensive, especially when tools have real-time constraints or hardware-specific action spaces. If a reviewer can turn a rich training-time stack into student-native demonstrations, a smaller deployment stack may preserve more of the useful behavior.

Physical systems also sharpen two assumptions. First, review must account for irreversible consequences. A Bash check can be replaced or retried cheaply; a robot action may change the world before its outcome can be inspected. Second, the student must be able to observe the evidence that justified a correction. A private guide cannot rely on privileged state that will be unavailable at deployment. Applying Harness-Zero to robotics therefore needs simulation, permissions, and observation contracts that are stricter than those in text-only tool use.

The review process also creates a natural preference signal. Each replacement compares a rejected response and a preferred response at the same state. The current method uses only the accepted trajectory for response-level SFT and discards the rejected side. The authors propose preference learning as future work. This is a credible extension because the data already contain a comparison induced by the guide, and preference optimization may extract more information than positive imitation alone.

The recursive-self-improvement framing is attractive but carries a supervision risk. Better models can design better harnesses, and those harnesses can generate more training data, but a reviewer’s systematic mistakes become high-confidence labels. The released code reduces accidental leakage by filtering failures and private markers; it cannot prove that the reviewer’s judgment is always correct. Weak-model experiments already show aggressive replacement becoming harmful.

Limitations: Capability, Latency, Deep Knowledge, and Expressiveness

The first limitation is reviewer capability. The main inference experiment uses strong models, but Appendix F shows the advantage shrinking on weaker ones. DeepSeek-V4-Flash scores 33.0 under h, 58.0 under h*, 32.3 with empty review, and 57.0 with evolved K. Qwen3.6-35B-A3B scores 16.0, 48.0, 12.0, and 36.0 respectively. The weakest reviewer replaces 66.0% of reviewed steps, yet its edits are net harmful. Reviewing is itself an agent capability.

The second limitation is collection cost. Every proposal is reviewed before execution, so each step needs at least one student call and one reviewer call. The review processes the trajectory prefix and reference harness and must finish before execution can continue. On USPTO, mean trial time rises from 100.1 seconds to 237.2 seconds, about 2.4 times the minimal harness. This cost disappears after distillation, but data collection can still be expensive for long or high-stakes tasks.

The third limitation is incomplete transfer of domain knowledge. USPTO improves to 30.0 but remains below the 38.0 achieved with h*. Procedures such as validation are easier to demonstrate than reaction priors and candidate generation. The paper suggests more trajectories, broader pretraining, or mid-training as possible remedies. The result should temper claims that a few hundred reviewed rollouts can internalize an entire scientific specialty.

The fourth limitation is representational. Context management, internal routing, and other runtime mechanisms may not correspond to a student response. An adapted guide can communicate when such behavior is relevant, but a response replacement cannot directly encode the mechanism’s implementation. The deployment harness must retain some capabilities even when the specialized source harness is removed.

There is also an empirical uncertainty that the authors do not quantify. Results are single-run, and the main tables report no confidence intervals or repeated-seed variance. SpreadsheetBench and USPTO contain 100 test tasks, so one task is one percentage point; AppWorld has 56 scenarios. Reviewer sampling can create additional variance during collection. The overall direction is supported, but the 2.6-point advantage of 44.3 over 41.7 should be replicated before treating it as a stable ranking.

Finally, the method assumes that a reviewer can make useful counterfactual judgments from the visible trajectory and K. It must predict what would happen if the original response were executed and whether a replacement would lead to a better state. In complex environments, that prediction can be the hardest part of the task. The paper identifies agent world models as a possible foundation for stronger review, but this remains outside the demonstrated system.

Conclusion: Move Part of the Harness from Deployment into the Weights

Harness-Zero’s contribution is a mechanism for converting cross-harness supervision into deployable policy. An evolved harness is adapted into a private guide, a reviewer corrects responses in the target action space, and SFT learns from the accepted trajectories. The deployed student keeps h but drops h*, K, and the reviewer.

The experiments support the claim with several checks. Empty review adds only 0.6 points, so the guide matters. Agent-as-harness with evolved guidance averages 81.1 before training, above code-as-harness at 78.1. Distillation raises the 9B student from 23.3 to 44.3 and exceeds the 41.7 obtained with h*. The USPTO result, where the distilled model remains below h*, prevents an overgeneralization: procedures transfer more readily than deep domain knowledge.

For future work, the clearest technical opportunities are selective review, preference learning from rejected replacements, and support for runtime mechanisms that cannot be expressed as a complete response. For robotics, the next question is whether the approach survives irreversible actions, noisy observations, and hard latency limits. If those constraints can be handled, harness optimization becomes more than a deployment configuration. It becomes a way for external systems to leave durable, testable capability in the model itself.

Golden quote: The harness worth carrying into deployment is not the expert system that remains attached to the model, but the behavior that system leaves inside the weights.

Agent HarnessHarness DistillationSupervised Fine-TuningLLM AgentAgent Training

SOURCE LINKS

https://arxiv.org/abs/2609.24974https://github.com/metaevo-ai/harness-zero

Related Papers

The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery

The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery

Modern coding agents are fundamentally reshaping the role of human input for computational research: they write code, launch in silico experiments, analyze their results, and autonomously operate with ever-increasing time horizons. Despite these developments, fully autonomous research agents suffer from a limited ideation radius and myopic hill-climbing search dynamics. To bridge this gap, we introduce The Agentic Scientist, an end-to-end distributed pipeline for human researchers to conduct computational research alongside modern coding agents. Building on its predecessor, our system decomposes research into task drafting, idea generation, planning, sandboxed experimentation, result aggregation, baseline and ablation construction, post-hoc analysis, and report writing. Importantly, each step in the pipeline is powered by the next generation of coding agents and leverages agentic self-verification. The overall research process is driven by a meta-controller that manages parallel research agents and enforces rubric verification at each stage. This design enables the same orchestration layer to be reused across heterogeneous domains with different execution environments, agentic coding harnesses, and evaluation metrics. Our system can be used across a spectrum of autonomy: either as a co-scientist tool or to perform end-to-end scientific investigation. Within the co-scientist setting, we leveraged a development version of our system to submit a single research paper, which got accepted for publication at ALIFE 2026. We further apply The Agentic Scientist fully autonomously to open problem-solving tasks in machine learning, and demonstrate the system's capability of scientific discovery across different domains of computational research. Our ablations suggest that reliable scientific autonomy depends not only on base model capabilities, but also on the cross-agent orchestration layer.

Robert Tjarko Lange, Luca Grillotti, Yutaro Yamada · 406 views · Oct 13, 2026
AI ScientistCoding Agent
Oct 13, 2026
SuperNav: An Agentic Navigation System for Any Task in Any Scene

SuperNav: An Agentic Navigation System for Any Task in Any Scene

General-purpose service robots need navigation systems that can handle diverse human requests in unfamiliar environments, combining task generality with scene generality. Some existing methods fine-tune multimodal large language models (MLLMs) to predict navigation actions, making their behavior dependent on the coverage of navigation training data and potentially limiting generalization to new requests and environments. Our key insight is to let the MLLM focus on interpreting requests, understanding scenes, and making decisions while preserving its general-purpose capabilities and delegating motion execution to navigation tools. To realize this idea, we introduce SuperNav, which equips a pretrained MLLM with a specialized agent harness without navigation-specific fine-tuning of the MLLM. Our harness supports these decisions with Navigation Skills, agent-oriented Tools for physical interaction, and task-progress and context management. A unified visual-point interface connects decision-making to motion by allowing the model to specify destinations directly in images and revise its decisions from execution feedback. Together, these components support sustained navigation across different task requirements and environments. SuperNav outperforms four evaluated baselines on instance-level, multi-object, and demand-driven tasks. Category-level evaluation on HM3D and deployment on a real quadruped robot further demonstrate its applicability across environments. Project Page: https://zju3dv.github.io/SuperNav/

Jinkai Zhang, Jingyi Xu, Yuanhong Yu · 366 views · Oct 8, 2026
Object NavigationEmbodied Agent
Oct 8, 2026
LongHarness: Stateful Evidence Management for Agentic Reasoning over 100M-Token Corpora

LongHarness: Stateful Evidence Management for Agentic Reasoning over 100M-Token Corpora

Corpus-level long-context reasoning requires agents to acquire and integrate evidence across document collections that exceed their context windows. Existing agent harnesses enable incremental access to these corpora, but acquired evidence and evolving conclusions often remain entangled in a growing interaction history. This implicit state management can lead to two coupled failure modes: unbounded exploration, where retrieval expands the interaction history without resolving decision-relevant information gaps, and evidence-state drift, where previously acquired evidence or subsequent corrections fail to inform the current conclusion. We present LongHarness, a model-agnostic harness that makes the interaction between acquired evidence and evolving conclusions explicit. Its core mechanism, Evidence-Conclusion Coevolution, uses unresolved conclusions to guide exploration toward missing information, while newly acquired evidence strengthens or revises the conclusions that directed the search. LongHarness couples this process with Evidence Lifecycle Management, which keeps accumulated evidence current and traceable throughout the interaction. Together, these mechanisms focus retrieval on remaining information needs and keep conclusions aligned with current evidence. We evaluate LongHarness on OfficeQA Pro V1 and V2, AA-LCR, and LongBench v2 using GLM-5.3-Flash and Qwen3.8-Flash. Compared with direct reasoning using the same backbone, LongHarness improves GLM-5.3-Flash exact match on OfficeQA Pro from 15.8% to 60.2% on V1 and from 4.4% to 61.1% on V2. Against state-of-the-art agent harnesses, it surpasses Claude Code by 12.0 percentage points on V1 and Codex by 18.9 points on V2. With Qwen3.8-Flash, LongHarness reaches 61.7% on V1, outperforming Claude Code by 8.3 points and Codex by 12.0 points.

Songtao Huang, Zhijie Zhong, Xiaohan He · 399 views · Oct 9, 2026
Agent HarnessLong-Context Reasoning
Oct 9, 2026
Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness

Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness

Recent game design agents have made substantial progress in generating playable games. However, program correctness does not ensure an enjoyable experience for players. We present Recursive Game Creator, an experience-oriented harness to advance agentic game development from rough game prototypes into entertaining games. Recursive Game Creator organizes recursive development around four components: Designer, Builder, Player, and Reviewer. The Designer translates user instructions and Reviewer's feedback into detailed plans. The Builder turns these plans into candidate games. The coding-native Player creates and executes reusable policies through programmatic interfaces to efficiently collect diverse gameplay trajectories, mitigating evaluation bias caused by slow GUI-based collection. The Reviewer uses carefully designed trajectory-based metrics to induce player preferences, integrating with visual evidence and explicit textual preferences to evaluate games against game-specific criteria. Finally, the Reviewer accepts the better version and provides improvement reviews for the next round, closing the recursive loop. Our method achieves state-of-the-art overall performance of 77.89 on GameCraft-Bench. On GameASG-Bench, it achieves a strict task success rate of 53.2%, a 34.1% improvement over the same-model baseline, and the highest mean runtime-check pass rate at 93.4% among compared methods. A user study shows longer playtime and higher ratings. Code is coming soon.

Jiajun Chen, Haoyu Wu, Mingda Jia · 348 views · Oct 6, 2026
Agentic Game DevelopmentRecursive self-improvement
Oct 6, 2026

As an Amazon Associate, we earn from qualifying purchases.