PAPER DEEP DIVE
Harness-Zero: Harness Distillation via Agent-as-Harness
Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and available information, so guidance from the optimized harness cannot serve directly as supervision for the target one. We introduce Harness-Zero, which enables harness distillation through agent-as-harness. Guided by the optimized harness, a harnessing agent corrects student responses before execution in the target harness's action space, turning harness guidance into training demonstrations. Fine-tuning on the resulting trajectories internalizes harness-induced behavior into the model, so the specialized harness can be removed at deployment. Our experiments spanning knowledge work, tool use, and science domains show that: (1) For frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness. (2) With the specialized harness removed at deployment, Harness-Zero improves the base model's macro-average task success from 23.3% to 44.3%, even exceeding the 41.7% it reaches with that harness still attached. (3) Harness-Zero recovers harness-induced behaviors absent from the base model, with 82.3% average recovery across 28 patterns in the three domains.
One-sentence summary: Harness-Zero uses a private guide adapted from an optimized agent harness to review and correct every unexecuted student response, then performs supervised fine-tuning on the accepted trajectories so the optimized harness behavior survives inside the model when only a minimal harness is deployed.
Why Harness Capability Is Usually Trapped Outside the Model
A language-model agent is not defined by its weights alone. The surrounding harness determines which tools the model can call, how observations are formatted, which middleware can block an action, what skills and memory are available, and when the interaction loop terminates. The paper uses the term agent harness for this external system. Its influence is large enough that the same base model can exhibit very different success rates under different harnesses, making harness design part of the agent’s effective capability.
This creates a deployment problem. A harness optimized for spreadsheets may be poor for multi-application workflows. A harness tuned around one model may become brittle after the model is updated. An agent intended to handle many domains can either use one compromise harness or route among an expanding set of specialized systems. The first choice caps performance; the second raises deployment complexity, maintenance cost, and context-switching risk.
Prior work follows two broad strategies. Code-as-harness optimizes prompts, tools, and middleware directly, and then deploys that optimized code around the model. Model training improves the policy inside a chosen system but does not explicitly transfer behavior discovered through harness optimization. Harness-Zero asks whether the gain of an optimized harness can become training signal rather than remaining a permanent deployment dependency.
The problem is not ordinary imitation learning. A trajectory collected under the optimized harness may contain tools absent from the target harness, or observations that the student is not allowed to see at deployment. Direct cloning can therefore teach commands that cannot execute or create dependence on information that will disappear. The technical task is to convert harness guidance across action spaces while preserving the procedure that made the source harness effective.
The authors call this problem agent harness distillation. The optimized harness is available during training, while a fixed target harness remains at deployment. The resulting behavior is stored in model parameters. This framing connects two normally separate forms of agent engineering: improving the external scaffold and improving the internal policy. It also states the scope precisely. Harness-Zero removes a specialized harness, not the need for every harness.
Notation: Three Harnesses with Different Jobs
The target harness h is the fixed deployment system. In the experiments it is a minimal mini-SWE-agent-style harness with a fixed system prompt and one Bash execution tool. The optimized student-side harness h* is evolved on training tasks and may contain additional tools, middleware, skills, and memory. The private reference harness K is adapted from h* so that a reviewing agent can use it. The three objects are related, but they are not interchangeable.
Let πθh denote the response policy of a model with parameters θ while operating under h. Let Th(c,y) denote the transition that executes response y from context c through h and returns the next student-visible context. This notation separates generated text from environmental execution. It also makes action-space mismatch explicit: a model can emit a string, but only the harness can decide whether that string is executable and how its result appears in the next context.
Agent-as-harness does not replace the sandbox with another free-form agent. It inserts a private review boundary around each student response. The student keeps acting under h, and h still executes the accepted action. The reviewer reads guidance adapted from h* and decides whether to pass or replace the proposal. The source harness can therefore shape the rollout without injecting inaccessible tools or hidden observations into the deployed student context.
The Three-Stage Method
The first stage evolves h* on training tasks and adapts it into K relative to h. The second stage collects trajectories by letting the student propose responses under h and asking the reviewer to intercept every response before execution. The third stage filters successful trajectories and fine-tunes the student on the accepted responses. At deployment, h*, K, and the reviewer are removed. Only the distilled policy and h remain.
flowchart LR
A["Training tasks D_train"] --> B["Evolve optimized harness h*"]
B --> C["Adapt h* against target h"]
C --> D["Private review guide K"]
E["Student proposal y_t under h"] --> F{"Review before execution"}
D --> F
F -->|PASS| G["Accept original y_t"]
F -->|REPLACE| H["Build valid z_t under h"]
G --> I["Execute only accepted response"]
H --> I
I --> J["Return student-visible observation c_t+1"]
J --> E
I --> K["Keep successful trajectories"]
K --> L["SFT on accepted responses"]
L --> M["Deploy student with h only"]
Figure 1 presents the same three stages. The left panel evolves the expert harness and translates its components into a private reference. The center panel shows one student rollout wrapped by a reviewer that can pass or replace each proposal. The right panel masks reviewer-perspective reasoning, trains on retained trajectories, and deploys without the training-time systems. The examples at the bottom show that review is not synonymous with constant intervention. A student that has already inspected a workbook can receive PASS, while a premature completion can be replaced by a response that continues the required file operations.
Figure 1: The Harness-Zero pipeline for harness evolution, reviewed rollout collection, SFT, and minimal-harness deployment. Image source: paper Figure 1.
Stage One: Evolve a Harness, Then Adapt It for Review
Given a training set Dtrain, the first stage performs two operations:
$$ h^{\star}=\operatorname{Evolve}(\mathcal{D}_{\mathrm{train}}), \qquad \mathcal{K}=\operatorname{Adapt}(h^{\star};h). $$
The first equation produces the optimized student-side harness. The second converts it into a guide for the reviewer. Adaptation depends on h because every correction ultimately has to be expressed and executed in the target action space. By keeping the two operations separate, the method avoids assuming that the expert harness and deployment harness share tools or observations.
The implementation uses a skill-guided evolution loop that analyzes recurring failures and updates a domain harness in three rounds. The resulting h* follows the DeepAgents abstraction, with tools, middleware, skills, and memory. This optimizer is not the main contribution and can be replaced by another automated harness-improvement method. Harness-Zero begins from the optimized system and studies how its induced behavior can become training data.
Adaptation changes both the audience and the enforcement point of each component. A source tool becomes a specification for constructing a student-native equivalent. Source middleware becomes review middleware that privately alerts the reviewer when a relevant condition appears. Skills and memory become diagnostic criteria and intervention guidance. A domain prompt may also tell the reviewer which failure patterns deserve particular attention.
These transformations preserve the intent of the optimized harness while producing responses that h can execute. The reviewer’s information access is intentionally narrow. It can read K, but it cannot inspect hidden solutions, verifier feedback, or environmental state beyond the student-visible context. If the reviewer needs evidence, it must propose an action available under h, such as a validation command, and let the result enter the student-visible trajectory. Corrections are therefore grounded in observations that the student can later reproduce.
Stage Two: Review at the Response Boundary
Code-as-harness runs the base model directly under the evolved harness:
$$ y_t\sim\pi_{\theta_0}^{h^{\star}}(\cdot\mid c_t), \qquad c_{t+1}=T_{h^{\star}}(c_t,y_t). $$
Both generation and execution are shaped by h*. The benefit remains tied to that system at deployment. Agent-as-harness instead starts with a proposal from the student under the target harness:
$$ y_t\sim\pi_{\theta_0}^{h}(\cdot\mid c_t). $$
The reviewer receives the visible context, the unexecuted proposal, the private guide, and its own history of earlier review decisions:
$$ (d_t,z_t)\sim\pi_H(\cdot\mid c_t,y_t,\mathcal{K},r_{{<t}}). $$
Here dt is PASS or REPLACE, zt is a complete replacement when one is needed, and r<t contains earlier private review exchanges and relevant reads from K. The accepted response is selected as follows:
$$ \tilde y_t= \begin{cases} y_t,&d_t=\textsc{pass}\\ z_t,&d_t=\textsc{replace} \end{cases} $$
Only the accepted response reaches the environment:
$$ c_{t+1}=T_h(c_t,\tilde y_t). $$
This distinction is central. Review is not a side conversation that later becomes extra context. It happens immediately before execution, and only the accepted response participates in the student-visible trajectory. The rejected proposal and the reason for rejection remain private. The resulting training sequence therefore has the same call-and-observation structure that the distilled model will encounter at deployment.
The reviewer is instructed to change as little as possible. It passes sound proposals unchanged and, when intervention is necessary, produces the smallest coherent full response that follows K. Full-response replacement matches the granularity of an agent turn: a turn can contain visible text, reasoning, and a tool call that must remain mutually consistent. The cost is coarse control, which the authors identify as a limitation and a target for future token-level or latent interventions.
One reviewing session spans the entire rollout. At each turn, the reviewer sees the student-visible events added since the previous review and the current proposal. This gives the correction enough local history to depend on state while avoiding a fresh replay of the whole environment at every step. A command can pass in one state and need replacement in another because the guide is interpreted against the current trajectory rather than compiled into static rules.
What the Repository Reveals About PASS and REPLACE
The released implementation makes the review boundary a typed middleware rather than a loose prompting convention. In src/harness_zero/student.py:98-208, HarnessReviewMiddleware wraps the model call, serializes the candidate response, validates its identifier, enforces turn and replacement budgets, and selects either the original message or a replacement. The decisive branch is compact:
if submission.decision == "PASS":
accepted = candidate.original
accepted_message = original_message
else:
accepted = submission.replacement
assert accepted is not None
accepted_message = accepted.to_message(candidate_id=candidate_id)
self._replacement_count += 1
self._store.append_review(candidate, submission, accepted)
Source: src/harness_zero/student.py:154-171. The environment receives a reconstructed accepted message, not arbitrary text returned by the reviewer.
The schema in src/harness_zero/review.py:15-96 also prevents ambiguous decisions. PASS cannot carry a replacement, REPLACE must include one, and a replacement must contain either visible content or a tool call. One submission is bound to one candidate. These constraints rule out partial patches, empty actions, and responses that claim to replace a candidate without supplying an executable turn.
if self.decision == "PASS" and self.replacement is not None:
raise ValueError("PASS must not include a replacement")
if self.decision == "REPLACE" and self.replacement is None:
raise ValueError("REPLACE requires a complete replacement")
if self.replacement is not None and not self.replacement.is_actionable:
raise ValueError("a replacement must contain visible content or a tool call")
Source: src/harness_zero/review.py:65-72. A replacement must be an actionable response, not merely an instruction to try something else.
Stage Three: Learn the Procedure Without Learning the Reviewer’s Role
After collection, the accepted responses become supervised targets. For retained trajectories Dreview, the objective is:
$$ \hat\theta= \arg\min_\theta-\sum_{\tau\in\mathcal{D}_{\mathrm{review}}} \sum_{t=1}^{T_\tau} \log\pi_\theta^h(\tilde y_t\mid c_t). $$
The model is not trained to emit h* actions and is not given the reviewer’s private context. It learns to produce accepted responses from student-visible ct under h. The labels include both proposals that passed unchanged and proposals that were replaced. This combination preserves behavior the student already performs well while inserting optimized procedures where the student alone would miss them.
A subtle failure mode is reviewer leakage. A replacement can contain reasoning such as “this candidate is incomplete” or “I should replace the student’s proposal.” Those statements make sense only from the review role. If copied into an SFT target, the distilled student may comment on a nonexistent candidate at deployment or act as if it has private review authority. The data pipeline addresses this in src/harness_zero/sft.py:12-77 by retaining only successful, exception-free trials and identifying replacement turns whose reasoning contains review-perspective language. The training code masks those reasoning tokens from the loss.
if reward < reward_threshold or exception is not None:
skipped_trials += 1
continue
events = [
json.loads(line)
for line in events_path.read_text(encoding="utf-8").splitlines()
if line.strip()
]
Source: src/harness_zero/sft.py:20-29. Failed or exceptional trials never become positive demonstrations.
src/harness_zero/dataset_utils.py:15-113 goes further by asserting that each serialized conversation contains only student-side messages. Private component paths, candidate files, the review submission tool, and review middleware names are rejected before training. This turns the information boundary into a data-construction invariant. Privacy is enforced by removing teacher-side structure from the sample, not by hoping that fine-tuning will cause the model to forget it.
The training recipe is sized for multi-turn tool use. Qwen3.5-9B is fine-tuned with LoRA rank 32, peak learning rate 2×10-4, final learning rate 1×10-6, batch size 8, maximum sequence length 65,536, and two epochs. The retained data contain 487 trials for SpreadsheetBench, 282 for AppWorld, and 500 for USPTO. The long context limit matters because accepted responses can include commands, file contents, and execution feedback rather than short question-answer pairs.
Deployment then follows a deliberately simple transition:
$$ y_t\sim\pi_{\hat\theta}^{h}(\cdot\mid c_t), \qquad c_{t+1}=T_h(c_t,y_t). $$
The target harness is unchanged; only the policy parameters differ. The model has internalized procedures that previously required the combination of an optimized harness and a reviewer. The deployed tuple is (πθhat, h), with h*, K, and the reviewer absent.
Experimental Design: Procedures, Interactive Tools, and Domain Knowledge
SpreadsheetBench Verified contains 400 real spreadsheet tasks. The authors use 300 for harness evolution and trajectory collection and hold out 100 for evaluation. The domain stresses state inspection, local edits, formula manipulation, and verification of the delivered file. Because all actions pass through a Bash tool, the harness must turn an underspecified request into a sequence of executable file operations.
AppWorld merges official train and development tasks into a 147-task training split and evaluates on 168 test_normal tasks grouped into 56 three-task scenarios. It requires interactive tool use across simulated applications. The metric is scenario goal completion rather than syntactic correctness of one API call. Success therefore depends on maintaining state and moving information across application boundaries.
USPTO Retrosynthesis uses 500 training tasks and 100 disjoint test tasks balanced over ten reaction classes. The model must predict single-step precursors. This setting combines executable procedures with chemical priors, candidate generation, and SMILES validation. It provides a useful contrast: a procedure can be demonstrated in a response, while a deeper knowledge prior may require much broader training coverage.
The inference comparison uses GPT-5.6 Sol and DeepSeek-V4-Pro. The distillation experiments use Qwen3.5-9B as the student and GPT-5.6 Sol as the reviewer. Harness evolution uses Kimi K3. Reasoning is enabled, with high effort when supported. In the training-free comparison, the same model can act as both student and reviewer, so the observed gain cannot be explained solely by inviting a stronger model to supervise a weaker one.
Result One: Extra Review Is Nearly Useless Without the Adapted Guide
Table 1 compares four conditions: the minimal harness h, the evolved harness h*, an agent reviewer with an empty guide, and an agent reviewer with K. Their averages are 68.6, 78.1, 69.2, and 81.1. Empty review improves the minimal harness by only 0.6 points. Another model call and a generic request to check the proposal do not create a reliable gain.
| Condition | Average success | Change over h | Interpretation |
|---|---|---|---|
| mini-SWE-agent under h | 68.6 | Baseline | One Bash action interface |
| meta-harness under h* | 78.1 | +9.5 | Use the evolved harness directly |
| Agent-as-harness, empty K | 69.2 | +0.6 | Review exists, but no expert guidance |
| Agent-as-harness, evolved K | 81.1 | +12.5 | Private guidance turns review into targeted correction |
The averages aggregate six benchmark-model cells in the paper’s Table 1. Agent-as-harness with K is at least as good as h in every cell and improves the average over h* by 3.0 points. The pattern is not a uniform mechanical lift. On DeepSeek-V4-Pro AppWorld, h* scores 91.1 and agent-as-harness scores 89.3. The same model gains from 77.0 to 86.0 on SpreadsheetBench and from 56.0 to 60.0 on USPTO. The reviewer interprets the guide in context rather than reproducing one fixed harness trajectory.
The comparison also suggests a model-update advantage. A code harness encodes assumptions about how a model should act, and those assumptions can age when the base model changes. Agent-as-harness keeps that adaptation in the reviewing model, which reads the guide against the current trajectory. A more capable model may therefore apply the same concept more effectively. The advantage is conditional: the experiments later show that weak reviewers can make harmful decisions.
Figure 2: Paper Table 1, comparing code-as-harness and agent-as-harness before any model training. The public copy is scaled to the site image-width policy.
Result Two: Distillation Beats Keeping the Expert Harness
Table 2 provides the main evidence for harness distillation. Qwen3.5-9B under h succeeds on 23.3% of tasks on average. Mounting h* raises the result to 41.7%. After Harness-Zero fine-tuning, the same base model under h alone reaches 44.3%. The distillation produces a 21.0-point absolute gain and a 90.1% relative gain over the untrained student, and it slightly exceeds the result of retaining the expert harness.
| Setting | SpreadsheetBench | AppWorld | USPTO | Average |
|---|---|---|---|---|
| Base model under h | 31.0 | 26.8 | 12.0 | 23.3 |
| Base model under h* | 39.0 | 48.2 | 38.0 | 41.7 |
| Base model with DeepAgents | 35.0 | 19.6 | 7.0 | 20.5 |
| Base model with Claude Code | 31.0 | 10.7 | 6.0 | 15.9 |
| Harness-Zero under h | 44.0 | 58.9 | 30.0 | 44.3 |
The domain breakdown exposes different transfer dynamics. On SpreadsheetBench, the distilled model reaches 44.0 versus 39.0 with h*. On AppWorld it reaches 58.9 versus 48.2. These improvements fit the method: inspecting state, making a local change, verifying the result, and carrying information across tools are procedures that can be demonstrated in student-native turns and learned by SFT.
USPTO behaves differently. The distilled model improves from 12.0 to 30.0 but remains below h* at 38.0. The expert harness supplies reaction priors, candidate-generation logic, and executable SMILES validation, not only an action order. Response-level demonstrations expose part of that knowledge, but the student does not absorb all of it from the collected trajectories. The paper therefore separates procedural behavior, which transfers well, from deep domain knowledge, which may require broader pretraining, mid-training, or more data.
Generic harnesses do not provide a shortcut. DeepAgents averages 20.5 and Claude Code averages 15.9, both below the minimal h. The 9B student struggles with larger tool suites and longer contexts, while AppWorld and USPTO need constraints that broad tool collections do not supply. More available tools can increase distraction when the model lacks a domain procedure for using them. This result supports the paper’s distinction between adding a general scaffold and distilling a successful domain-specific one.
Figure 3: Paper Table 2, comparing the base policy, expert and generic harnesses, and the distilled policy across three domains.
Result Three: Supervision Must Match Both the Action Space and the Desired Behavior
The USPTO ablation is designed to separate “more successful data” from “the right kind of successful data.” Teacher rollouts succeed during collection 52.0% of the time but produce a distilled pass@1 of only 12.0%. A teacher operating under h* reaches a 62.0% collection rate but distills to 3.0%. The student running under h* collects at 39.4% and still tests at 12.0%.
| Trajectory source | Collection success | Distilled test pass@1 | Change from base |
|---|---|---|---|
| Base model, no training | — | 12.0 | 0.0 |
| Teacher rollout | 52.0 | 12.0 | 0.0 |
| Teacher under h* | 62.0 | 3.0 | -9.0 |
| Student under h* | 39.4 | 12.0 | 0.0 |
| Review with empty K | 44.2 | 11.0 | -1.0 |
| Oracle answer | 98.6 | 15.0 | +3.0 |
| Harness-Zero with K | 59.4 | 30.0 | +18.0 |
The source with the highest collection rate performs much worse than Harness-Zero. Oracle answers almost always complete the task, but they teach the result rather than a procedure the student can execute under h. Teacher rollouts may demonstrate successful outcomes without exposing the model-specific path needed by the student. Teacher actions under h* can be even less compatible with the target action space. Harness-Zero does not maximize collection success; it maximizes the usefulness of each accepted turn as a target-harness demonstration.
The 30.0 result against alternatives from 3.0 to 15.0 is stronger evidence than the aggregate score alone. It shows that a successful trajectory is not automatically good supervision. The response must be executable by the student, grounded in visible observations, and shaped by the behavior that harness optimization discovered. This is why the private guide and the target-harness boundary are both necessary.
Figure 4: Paper Table 3, comparing alternative supervision sources by collection success and post-distillation USPTO performance.
Behavior Recovery Shows What the Weights Actually Learned
The authors also measure whether the distilled model recovers specific behaviors that the expert harness induces and the base student lacks. They retain 28 patterns: 18 in SpreadsheetBench, 6 in USPTO, and 4 in AppWorld. For each pattern, recovery is the fraction of support tasks on which Harness-Zero exhibits the behavior. The average recovery rate is 82.3%.
The construction makes the comparison meaningful. By definition, the untrained student under h exhibits each pattern on 0% of its support tasks, while the same student under h* exhibits it on 100%. Harness-Zero is evaluated under h alone. Its 82.3% recovery therefore measures behavioral transfer rather than final-answer memorization. The patterns include inspecting inputs, avoiding premature completion, verifying edits, and using domain-specific validation procedures.
The behavior analysis also clarifies what remains outside the method. Some harness mechanisms, such as context compaction or internal context switching, are not naturally expressible as a student response. A reviewer can describe when such a mechanism is needed, but a full-response replacement cannot directly carry the implementation. Harness-Zero narrows the functionality that a deployment harness must supply; it does not eliminate every runtime capability.
Discussion: The Main Shift Is in Where Training Signal Comes From
Harness-Zero turns harness optimization into a temporary data generator and reviewer. The optimized system remains an external object during collection, but its effect is converted into parameter updates. This is different from treating runtime engineering and model training as separate tracks. It makes the artifacts of runtime search reusable as policy supervision.
The idea is relevant to embodied agents because robot stacks also place perception filters, skill libraries, recovery logic, safety checks, and workflow code outside the foundation model. Maintaining a domain-specific stack for every task is expensive, especially when tools have real-time constraints or hardware-specific action spaces. If a reviewer can turn a rich training-time stack into student-native demonstrations, a smaller deployment stack may preserve more of the useful behavior.
Physical systems also sharpen two assumptions. First, review must account for irreversible consequences. A Bash check can be replaced or retried cheaply; a robot action may change the world before its outcome can be inspected. Second, the student must be able to observe the evidence that justified a correction. A private guide cannot rely on privileged state that will be unavailable at deployment. Applying Harness-Zero to robotics therefore needs simulation, permissions, and observation contracts that are stricter than those in text-only tool use.
The review process also creates a natural preference signal. Each replacement compares a rejected response and a preferred response at the same state. The current method uses only the accepted trajectory for response-level SFT and discards the rejected side. The authors propose preference learning as future work. This is a credible extension because the data already contain a comparison induced by the guide, and preference optimization may extract more information than positive imitation alone.
The recursive-self-improvement framing is attractive but carries a supervision risk. Better models can design better harnesses, and those harnesses can generate more training data, but a reviewer’s systematic mistakes become high-confidence labels. The released code reduces accidental leakage by filtering failures and private markers; it cannot prove that the reviewer’s judgment is always correct. Weak-model experiments already show aggressive replacement becoming harmful.
Limitations: Capability, Latency, Deep Knowledge, and Expressiveness
The first limitation is reviewer capability. The main inference experiment uses strong models, but Appendix F shows the advantage shrinking on weaker ones. DeepSeek-V4-Flash scores 33.0 under h, 58.0 under h*, 32.3 with empty review, and 57.0 with evolved K. Qwen3.6-35B-A3B scores 16.0, 48.0, 12.0, and 36.0 respectively. The weakest reviewer replaces 66.0% of reviewed steps, yet its edits are net harmful. Reviewing is itself an agent capability.
The second limitation is collection cost. Every proposal is reviewed before execution, so each step needs at least one student call and one reviewer call. The review processes the trajectory prefix and reference harness and must finish before execution can continue. On USPTO, mean trial time rises from 100.1 seconds to 237.2 seconds, about 2.4 times the minimal harness. This cost disappears after distillation, but data collection can still be expensive for long or high-stakes tasks.
The third limitation is incomplete transfer of domain knowledge. USPTO improves to 30.0 but remains below the 38.0 achieved with h*. Procedures such as validation are easier to demonstrate than reaction priors and candidate generation. The paper suggests more trajectories, broader pretraining, or mid-training as possible remedies. The result should temper claims that a few hundred reviewed rollouts can internalize an entire scientific specialty.
The fourth limitation is representational. Context management, internal routing, and other runtime mechanisms may not correspond to a student response. An adapted guide can communicate when such behavior is relevant, but a response replacement cannot directly encode the mechanism’s implementation. The deployment harness must retain some capabilities even when the specialized source harness is removed.
There is also an empirical uncertainty that the authors do not quantify. Results are single-run, and the main tables report no confidence intervals or repeated-seed variance. SpreadsheetBench and USPTO contain 100 test tasks, so one task is one percentage point; AppWorld has 56 scenarios. Reviewer sampling can create additional variance during collection. The overall direction is supported, but the 2.6-point advantage of 44.3 over 41.7 should be replicated before treating it as a stable ranking.
Finally, the method assumes that a reviewer can make useful counterfactual judgments from the visible trajectory and K. It must predict what would happen if the original response were executed and whether a replacement would lead to a better state. In complex environments, that prediction can be the hardest part of the task. The paper identifies agent world models as a possible foundation for stronger review, but this remains outside the demonstrated system.
Conclusion: Move Part of the Harness from Deployment into the Weights
Harness-Zero’s contribution is a mechanism for converting cross-harness supervision into deployable policy. An evolved harness is adapted into a private guide, a reviewer corrects responses in the target action space, and SFT learns from the accepted trajectories. The deployed student keeps h but drops h*, K, and the reviewer.
The experiments support the claim with several checks. Empty review adds only 0.6 points, so the guide matters. Agent-as-harness with evolved guidance averages 81.1 before training, above code-as-harness at 78.1. Distillation raises the 9B student from 23.3 to 44.3 and exceeds the 41.7 obtained with h*. The USPTO result, where the distilled model remains below h*, prevents an overgeneralization: procedures transfer more readily than deep domain knowledge.
For future work, the clearest technical opportunities are selective review, preference learning from rejected replacements, and support for runtime mechanisms that cannot be expressed as a complete response. For robotics, the next question is whether the approach survives irreversible actions, noisy observations, and hard latency limits. If those constraints can be handled, harness optimization becomes more than a deployment configuration. It becomes a way for external systems to leave durable, testable capability in the model itself.
Golden quote: The harness worth carrying into deployment is not the expert system that remains attached to the model, but the behavior that system leaves inside the weights.



