PAPER DEEP DIVE
HarnessMM: Recursive Self-Improvement for Harness Token Efficiency
Agent harnesses often invoke the reasoning model again after tool feedback even when an earlier call has already fixed the next action. HarnessMM reuses resolved decisions at the model-tool boundary while keeping actor weights fixed. Executable Decision Graphs retain exact actions, literal arguments, causal dependencies, and public guards; Guarded Decision Reuse activates a stored decision from observed receipts, and Certified Callback Substitution executes it through the native interface while recording removal of the registered slow callback. New semantic decisions, unsupported evidence, and selector abstention fall back to the original actor. A recursive self-improvement loop refines the companion harness configuration from training feedback, with validation selecting the deployed configuration. On HumanEval, Codex preserves 90.82% success while reducing tokens per task from 66.72k to 47.02k; DeepSeek improves from 71.43% to 85.71% while cost falls from 96.28k to 11.62k. A separate MBPP ablation reports 26.91% lower cost than single-use graphs and 19.52% lower cost than retained callbacks.
One-Sentence Summary
HarnessMM reduces agent-harness tokens by executing already-resolved tool actions after public receipts satisfy authored guards, while preserving the slow reasoning actor for new semantics, unsupported evidence, or selector abstention.
Research Context and Motivation
Tool-using agents repeatedly alternate reasoning, action, and observation. A coding actor may write a file, receive a successful build receipt, and then call the model again merely to choose a test command that was already obvious in the previous turn. The repeated call consumes the whole context and emits another reasoning trace even though the next action and its literal arguments did not change.
Current efficiency work often reduces the cost of a call rather than whether the call occurs. Prompt compression shortens input, observation masking removes trajectory history, thinking budgets limit output, and model routing chooses a cheaper reasoner. These mechanisms can help, but a short classifier output does not prove that the original reasoning callback disappeared.
The paper names the resulting accounting failure the Substitution Paradox. If a fast stage costs Cf and leaves a continuation costing Cn intact, the new route costs Cn+Cf. A useful replacement must remove more computation than it introduces, including authorship, selection, recovery, and added context.
HarnessMM therefore targets a narrow boundary in an existing agent harness. Model weights, tools, permissions, and task interfaces remain fixed. After a slow model has fixed an action and its exact arguments, the controller stores that commitment. Future feedback is used to decide whether the action is ready, not to regenerate its semantics.
This focus is relevant to embodied agents even though the paper does not run a robot-control experiment. Robot stacks contain many deterministic transitions: a scan leads to a known grasp, a grasp receipt activates a known lift, and a successful localization activates a predeclared inspection pose. Such transitions share the paper’s structure of resolved action plus a public readiness condition.
Formal Setup
For task x, harness h induces a finite computation graph Gh(x)=(V,E). Nodes may represent reasoning, retrieval, routing, tools, verification, or recovery. A policy π determines which nodes are invoked, and node v has model-token cost cv. Total expected cost is:
$$C(\pi)=\mathbb{E}\left[\sum_{v\in V}\mathbf{1}\{v\text{ invoked under }\pi\}\,c_v\right].$$
The indicator is the important object. HarnessMM changes whether a reasoning node appears, not merely how many tokens it emits. Deterministic tool nodes consume no actor tokens, but they affect future invocation probabilities and task quality, so quality must be measured on the complete task population.
For benchmark b, let Qb be the official score and let Rb(π)=Qb(h0)-Qb(hπ) be capability regret relative to the original harness. The optimization problem is:
$$\min_{\pi} C(\pi)\quad\text{subject to}\quad R_b(\pi)\leq\epsilon_b.$$
Zero-regret protocols set ϵb=0. Failed tasks remain in both the numerator and denominator. This constraint rules out a misleading claim that fewer model calls are automatically better if task success falls outside the declared tolerance.
Method
Executable Decision Graphs preserve commitments
At slow decision i, actor H0 may author a bounded executable continuation Gi=(Vi,Ei). The future bank is Bi={d1,...,dm}. Each decision node contains an action, literal arguments, a predecessor, and a public guard:
$$d_j=(a_j,\mathrm{arg}_j,\mathrm{pred}_j,g_j).$$
An EDG is not a natural-language plan. It stores the native tool name and exact arguments, so a selector cannot synthesize new values. The author also fixes the guard gj:Spublic→{true,false}. The runtime evaluates that condition against observed state and receipts; it cannot rewrite the guard to make a candidate pass.
Dependencies must form an acyclic graph. A missing receipt, false or unknown guard, incomplete predecessor, or argument that requires fresh inference sends control back to the slow actor. The graph is bounded because unconstrained continuation generation could turn one expensive call into a larger and less reliable authoring problem.
Guarded Decision Reuse separates readiness from semantics
Guarded Decision Reuse, or GDR, retains the graph across tool transitions. For node dj and public state st, eligibility is the product of three checks:
$$\begin{aligned} \operatorname{eligible}(d_j,s_t) ={}&\mathbf{1}\{\mathrm{pred}_j\text{ completed}\}\\ &\cdot\mathbf{1}\{g_j(s_t)=\mathrm{true}\}\\ &\cdot\mathbf{1}\{\mathrm{arg}_j\text{ closed}\}. \end{aligned}$$
The first factor preserves causal order, the second checks observed evidence, and the third prevents a stored template from concealing an unresolved argument. Guards consume receipts that have already happened. They cannot inspect predicted tool output or hidden future state.
A unique eligible node can execute deterministically. With multiple eligible nodes, a Jev-inspired typed selector returns one candidate identifier or abstains. It has no interface for changing arguments or inventing a new action. An empty candidate set or abstention invokes the original actor, which may perform new reasoning and author another graph.
This differs from a conventional skill library. A reusable skill usually captures a procedure intended to transfer across tasks. An EDG stores episode-specific decisions whose semantic choices are already resolved. The mechanism does not assume that a successful procedure generalizes; it assumes only that this task has an exact executable commitment.
Certified Callback Substitution removes a registered call
Certified Callback Substitution, or CCS, dispatches an admitted action through the original native tool interface and omits the slow callback registered at that boundary. The parent route is st→H0→at. The replacement route is st→dj→at.
A certificate binds the graph hash, decision identifier, predecessor receipt, exact arguments, native tool receipt, and bypassed callback identifier:
$$\begin{aligned} \kappa_t=(&\mathrm{graph\_hash},\mathrm{decision\_id},\\ &\mathrm{predecessor\_receipt},\mathrm{exact\_args},\\ &\mathrm{tool\_receipt},\mathrm{bypassed\_callback\_id}). \end{aligned}$$
The paper also states the required callback-removal invariant directly:
$$N^{\mathrm{parent}}_{\mathrm{slow},t}-N^{\mathrm{HarnessMM}}_{\mathrm{slow},t}\geq 1.$$
This evidence is structural. It proves that the exact action ran and that the specified model callback did not occur. It does not prove semantic correctness, an identical counterfactual trajectory, or a higher task score. Those claims require complete native evaluation.
The implementation mirrors the distinction. In integrations/, localGuard rejects invalid guards, missing receipts, native errors, non-JSON evidence, or missing required fields before checking return codes, stdout, stderr, or structured JSON. A compact excerpt is:
export function localGuard(node, receipt) {
const guard = node.guard;
if (!guard || typeof guard !== 'object' || Array.isArray(guard) || !Object.keys(guard).length)
return {known:false, eligible:false, reason:'invalid_guard'};
let r=receipt?.value?.native_text ?? receipt?.value;
for(let i=0;i<3 && typeof r==='string';i++) {
try {r=JSON.parse(r);} catch {return {known:false,eligible:false,reason:'non_json_receipt'};}
}
if(!r || typeof r!=='object')
return {known:false,eligible:false,reason:'missing_receipt'};
if(receipt?.is_error)
return {known:true,eligible:false,reason:'native_error'};
}deterministicAdmission at lines 187-205 adds dependency completion, error risk, and a visible net-saving threshold. A candidate is not admitted merely because a guard matches; the implementation also estimates whether the fast path saves more than 1,024 tokens and otherwise defers to the slow actor.
The certificate is recorded in sar_v2.mjs:259-279. The following excerpt retains the identity fields that connect the graph, selected action, exact arguments, native result, and callback boundary:
this.record({kind:'sar/substitution-certificate',
schema:'certified-native-substitution/v1',
bank_hash:item.bank_sha256,
candidate_id:item.candidate.label,
...item.boundary,
exact_native_tool:e.name,
exact_arguments:e.arguments,
native_receipt_hash:hash(e.result),
slow_requests_at_boundary:0,
slow_sequence_before:item.slow_snapshot,
slow_sequence_after:this.slowSequence.length,
valid_substitution:true});The Codex adapter in phiagent/ performs the same conceptual loop through NativeBankBoundary. It registers one public native bank, executes the root, follows candidates by predecessor, records typed selection or abstention, and stops when the bank is exhausted or a native error occurs. It preserves the parent action space rather than granting new privileges.
Recursive self-improvement changes the surrounding harness
The RSI component does not train actor weights online. Training feedback proposes changes to author prompts, context serialization, adapters, and finalization rules. Candidate configurations are frozen for validation, a selected configuration is frozen before test, and native scoring evaluates the full population.
The paper separates two time scales. RSI changes the harness across rounds; GDR and CCS decide whether a reasoning callback occurs within an episode. A benchmark label containing the word HarnessMM does not by itself prove persistent execution. Some companion studies alter only instructions or prompts, and Appendix D.5 explicitly maps each reported configuration to its intervention.
flowchart TB
Slow[Slow actor H0] --> Author[Author exact actions and public guards]
Author --> Graph[Persistent executable decision graph]
Graph --> Root[Execute native root action]
Root --> Receipt[Observe public tool receipt]
Receipt --> Check{Guard known and true}
Check -- no or unknown --> Slow
Check -- yes --> Select{One or many eligible nodes}
Select -- unique --> CCS[Execute exact native action]
Select -- many --> Typed[Typed selector or abstain]
Typed -- eligible --> CCS
Typed -- abstain --> Slow
CCS --> Cert[Bind graph args receipt and bypassed callback]
Cert --> Receipt
CCS --> SlowNew[New semantic decision needed]
SlowNew --> Slow
Complete-task accounting prevents phantom savings
Episode cost sums actor input and output over every physical call:
$$C_{\mathrm{episode}}(h;x)=\sum_{c\in\mathcal{C}(h,x)}(T_c^{\mathrm{in}}+T_c^{\mathrm{out}}).$$
The set includes authorship, selection, remaining slow calls, finalization, recovery, fallback, and failed executions. Development cost is kept separately. If the resulting harness is deployed to N tasks, the amortized ledger is:
$$\overline{C}_{N}=\frac{C_{\mathrm{dev}}}{N}+\mathbb{E}_{x}[C_{\mathrm{episode}}(h;x)].$$
This distinction matters because an execution-time saving is not necessarily an all-in RSI saving. A method could reduce inference per task while spending more on proposals, validation, rejected candidates, and human engineering. phiagent/ also rejects a usage record whose total does not equal input plus output, preventing a simple aggregation mismatch.
Experimental Results
Industrial coding harnesses
On all 98 HumanEval test tasks, Codex preserves 90.82% success while falling from 66.72k to 47.02k tokens per task. DeepSeek Harness rises from 71.43% to 85.71% while falling from 96.28k to 11.62k. The DeepSeek paired outcomes contain 22 improvements and eight regressions, with a two-sided exact McNemar result of p=0.0161.
The authors are explicit that these are integrated-harness comparisons. They combine literal-bank reuse, context management, and finalization, so the total call reduction cannot be assigned entirely to callback substitution. The MBPP ablation is the cleaner mechanism test because those surrounding settings are held fixed.
| Population | Harness pair | Success and tokens | Certified substitutions |
|---|---|---|---|
| HumanEval, 98 tasks | Codex → HarnessMM | 90.82% → 90.82%; 66.72k → 47.02k | 18 total, 0.18/task |
| HumanEval, 98 tasks | DeepSeek → HarnessMM | 71.43% → 85.71%; 96.28k → 11.62k | 147 total, 1.50/task |
| MBPP-sanitized, 257 tasks | DeepSeek → HarnessMM | 90.27% → 91.83%; 53.98k → 10.61k | 472 total, 1.84/task |
| MBPP-sanitized, 257 tasks | Codex → HarnessMM | 91.83% → 90.66%; 129.55k → 49.22k | 158 total, 0.61/task |
The Codex MBPP pair loses 1.17 success points while using roughly 62% fewer tokens. The result is still economically interesting, but it is not a no-regret outcome. DeepSeek on HumanEval improves overall yet regresses on eight paired tasks. Reporting both task-level changes and total cost prevents a favorable mean from hiding operational failures.
The MBPP ablation isolates persistence and callback removal
Full HarnessMM reaches 93.77% on 257 MBPP-sanitized tasks at 13.17k tokens per task. Limiting each graph to one use, which removes GDR, yields 93.00% at 18.03k. Persistence therefore lowers complete cost by 26.91%. Retaining the slow callback while still executing the graph yields 93.39% at 16.37k, so callback removal lowers cost by another 19.52%.
Disabling graph execution but retaining authorship produces the uncomfortable control: 95.33% at 16.19k. HarnessMM spends 18.62% fewer tokens than this condition but solves four fewer tasks. The correct interpretation is not universal quality improvement. It is that persistent reuse and callback removal have measured cost benefits while the current authoring and guard policy still makes quality mistakes.
| MBPP condition | Success | Tokens/task | Interpretation |
|---|---|---|---|
| Full HarnessMM | 93.77% | 13.17k | Reference condition |
| No GDR, single-use EDG | 93.00% | 18.03k | Full method is 26.91% cheaper |
| No CCS, callbacks retained | 93.39% | 16.37k | Full method is 19.52% cheaper |
| No EDG execution | 95.33% | 16.19k | Full method is 18.62% cheaper but solves four fewer tasks |
The six companion panels have different causal targets
Figure 2 reports ALFWorld, SearchQA, LiveMath, AIME, Terminal-Bench 2, and GAIA2. These are not six repeats of the core EDG experiment. They use different task populations and mechanisms. ALFWorld primarily tests task-family instructions, LiveMath changes an answer-first prompt and output cap, AIME uses a frozen compiled decision policy, and GAIA2 combines a literal bank with context handling.
On the complete 2,133-task ALFWorld population, No Skill scores 42.29% at 34.05k, while HarnessMM’s selected instruction configuration scores 66.20% at 31.12k. On the 3,395-question SearchQA validation set, HarnessMM reaches 76.76 EM at 0.99k, compared with 69.81 EM at 1.23k for No Skill. Both are useful, but neither isolates persistent graph execution.
| Study | No Skill baseline | Selected HarnessMM configuration | Scope to retain |
|---|---|---|---|
| ALFWorld, 2,133 tasks | 42.29%, 34.05k | 66.20%, 31.12k | Instruction-content effect; local controller disabled |
| SearchQA, 3,395 questions | 69.81 EM, 1.23k | 76.76 EM, 0.99k | Routing, rule sentences, draft, and fallback |
| LiveMath, 160 questions | 19.38%, 4.10k | 29.38%, 0.83k | One-call prompt and output-limit change |
| AIME, 560 questions | 86.96%, 4.59k | 87.14%, 5.90k | Similar quality, but selected method costs more |
| Terminal-Bench 2 | 20.63%, 178.19k | 26.03%, 57.67k | Archived partial cohort, not a complete final test |
| GAIA2, 480 scenarios | 42.71%, 730.22k | 43.96%, 639.64k | Integrated context and action bank; below SkillOpt’s 46.88% |
AIME is a valuable counterexample. The selected HarnessMM policy costs 4.59k to 5.90k tokens per task. Its execution output falls only slightly while it adds 1.08k of shared authorship and 0.54k of input. Math performance is dominated by generated reasoning rather than repeatable native actions, exactly the setting where the substitution opportunity is weak.
GAIA2 lowers complete actor cost by about 12% relative to No Skill, but scores 43.96% against 46.88% for SkillOpt. HarnessMM performs 146 verified fast executions, only 1.40% of 10,424 action attempts, and input accounts for 99.47% of its actor cost. Most of the observed total-cost change must therefore be attributed to the surrounding context treatment, not the small number of certified substitutions.
RSI improves execution efficiency on MiniWoB++
Across eight training-guided iterations on the same 32 MiniWoB++ tasks, success rises from 34.38% to 90.63% and tokens per task fall from 3.26k to 1.06k. At iteration eight both arms solve 29 of 32 tasks, but persistence still saves 29.83%, from 1.51k to 1.06k.
The experiment uses Qwen3.5-9B, full screenshots, native rewards, 15 actions, and a 180-second episode limit. It counts physical model requests across all tasks and includes fallback and unsuccessful calls. Optimization and validation costs are separate, so the 67.43% reduction over the eight iterations is an execution result, not proof of an all-in RSI return.
Implementation Review
The repository makes the paper-to-code mapping unusually direct. cot_jev.mjs handles action-bank authoring and schema validation, sar_v2.mjs implements guards, persistent execution, selection, and certificates, and codex_native_app_bank.py adapts the loop to the Codex native boundary. deployment_token_accounting.py reconstructs physical provider usage rather than counting only successful outputs.
The local example node examples/guarded_reuse.mjs reports three certified substitutions, no slow calls during graph execution, and one slow call after the graph is exhausted. python examples/partition.py produces the synthetic 2/2/6 split. These checks demonstrate controller behavior, not benchmark performance. The full repository test suite was not run for this review.
The hardest integration surface is not hashing. It is defining a public state that contains enough evidence without leaking future outcomes. The adapter must also preserve exact native arguments, report errors, and correlate the omitted callback with a physical request ledger. A replay that fabricates a clean certificate would break the relationship between the proof and the actual execution route.
Limitations
The first author-stated limitation is opportunity dependence. Savings require actions that are already resolved and supported by public evidence. New semantic decisions still need slow reasoning, and authorship, selection, or recovery may cost more than the removed callback. Tasks whose observations change the meaning of every next action offer little reusable structure.
The second author-stated limitation is task-specific adaptation and uncertain policy quality. Public-state adapters are implementation-specific, and small development sets do not pin down policy-level quality differences. Provider accounting, context caching, and input truncation can also change token conclusions. A matching guard proves only its declared public condition, not general task correctness.
My main independent concern is label leakage in downstream summaries. HumanEval, ALFWorld, SearchQA, and GAIA2 all include companion configurations. Attributing their entire gain to GDR and CCS would overstate the core method. The paper’s own Appendix D.5 is careful about this, but short abstracts and social posts tend to preserve the largest percentage and discard the experimental boundary.
Amortization is a second operational concern. Development cost is divided by deployment volume N. A workflow executed only a few dozen times may not repay authoring, validation, and rejected candidates even when per-episode tokens fall. A high-volume production harness is a better fit. The paper states the accounting equation but does not choose a single commercial deployment scenario.
The appendix also contains important failures. A compact SearchQA context reaches 70.54 EM versus 75.46 for the later full-context configuration. A spreadsheet state controller reaches 0.00% versus 1.03% for its parent and costs more. An early Terminal-Bench subset scores 8.33% versus 25.00%, and a visual-web subset scores 25.00% versus 33.33%. GQA grounding and equal-length generic instructions show identical accuracy, while an MMMU instruction study cannot reach the 10% saving target even with zero output cost.
These negative results narrow the claim in a useful way. HarnessMM is not a universal token compressor. It is a boundary-level execution policy whose benefit depends on resolved action density, trustworthy receipts, exact native dispatch, and a complete accounting system.
Conclusion and Outlook
HarnessMM contributes an execution contract rather than a new actor model. EDG stores an exact commitment, GDR decides whether observed public state activates it, CCS replaces a registered reasoning callback with a native action, and complete accounting prevents authorship or recovery from vanishing from the comparison. The MBPP ablation supplies the clearest evidence that graph persistence and callback removal affect cost.
For embodied systems, the next step is to extend the contract to actions with continuous parameters and safety envelopes. A grasp pose, velocity, force, collision clearance, and sensor freshness may all need closed constraints before a fast path is admissible. A single successful receipt cannot establish long-term correctness under occlusion, latency, or distribution shift.
The most credible early deployment is deterministic robotics software: build and test commands, log checks, resource loading, device self-checks, and predeclared experiment sequences. These operations have exact interfaces and public receipts, and repeated use can amortize authoring cost. Open-world manipulation is better treated as a hybrid system in which HarnessMM accelerates only the resolved branches and returns to slow reasoning when semantics change.
A removed model call is a real optimization only when the action remains exact, the evidence remains public, and failure still returns to reasoning.
