PAPER DEEP DIVE
Self-Improvement Memory Enables Continual Recursive Self-Improvement
This paper introduces Continual Recursive Self-Improvement (CRSI) for agent harnesses and Self-Improvement Memory (SIM), a structured record that links recurring root causes to selected edits, rationales, and predicted or observed outcomes. Starting from a harness evolved on a source benchmark or model, CRSI resumes optimization after a dataset or model transition rather than restarting. Across six cross-dataset settings, full SIM has the highest final held-out score and beats harness-only transfer by 2.43 to 4.35 points; across four Opus-to-Gemini model-family transfers it beats the harness-only continuation by 3.60 to 11.33 points. A ten-run paired coding study reports a mean gain of 2.36 points over continuation without SIM (95% CI 1.05 to 3.68, p=0.003).
One-Sentence Summary
The paper transfers an evolved agent harness together with a structured history of why each edit was attempted and what happened, allowing recursive optimization to continue productively after the benchmark or policy model changes.
Why This Problem Matters
An LLM does not act through model weights alone. Its effective behavior is shaped by an agent harness that assembles context, exposes tools, controls execution, and decides when a task is complete. Prior work treats this harness as executable code around a fixed policy and improves it through repeated evaluation. The resulting process is recursive because feedback from one harness version determines how the next version is written.
Harness evolution can produce gains that survive on held-out tasks or transfer to another model. Most transfer procedures, however, carry only the final harness. The artifact shows what the winning code became, but not why a rule was introduced, which failures it was expected to fix, whether those predictions materialized, or which model and benchmark supplied the evidence. Once the target setting changes, later optimizers can misread the intent of inherited code.
A completion gate illustrates the failure mode. Source optimization may add a check that prevents an agent from stopping after a parsing error, an unexecuted command, or a visible terminal crash. In a new benchmark, the actual failure may instead be a feasible solution that is far from optimal. Carrying only the gate can encourage the next optimizer to harden the old checks. Carrying the gate's rationale and measured outcomes can show that execution verification remains useful while a new quality criterion must be added.
The paper names the broader setting Continual Recursive Self-Improvement (CRSI). In cross-dataset CRSI, the policy stays fixed and the benchmark changes. In cross-model CRSI, the benchmark and split stay fixed while the policy changes. Both cases resume optimization from the final source harness for a matched ten-round target run. The missing object is not another model parameter but a usable representation of the improvement process itself.
Self-Improvement Memory (SIM) fills that gap. It is distinct from task-solving memory and from a flat list of code diffs. SIM organizes attempts around recurring harness problems and links a root cause to selected edits, rationales, predictions, outcomes, and provenance. Both the agent that proposes harness changes and the agent that selects among them can consult the record.
Formal Setting
A harness optimization problem is defined by a policy model and a task collection:
$$\sigma=(M,\mathcal{X}),\qquad \mathcal{X}=\mathcal{X}^{\mathrm{evo}}\cup\mathcal{X}^{\mathrm{eval}}.$$
Here, $M$ is the fixed task-solving policy. The evolve split $\mathcal{X}^{\mathrm{evo}}$ is the only source of feedback available for proposing edits, selecting candidates, updating memory, or deciding when to stop. The held-out split $\mathcal{X}^{\mathrm{eval}}$ measures the starting and final harness but does not influence the search.
At round $t$, harness $H_t$ runs on the evolve split and produces scores, traces, and errors, denoted $F_t$. The ordinary run history $D_t$ contains earlier feedback and code changes. The proposer generates a candidate set from these inputs:
$$\mathcal{C}_t=P(H_t,F_t,D_t).$$
The selector screens those candidates and evaluates the ones that survive screening. If $F_t^{\mathcal{C}}$ is the resulting candidate feedback, the next retained harness is
$$H_{t+1}=\mathcal{S}(H_t,F_t,\mathcal{C}_t,F_t^{\mathcal{C}}).$$
When no proposal is accepted, the current harness is retained unchanged. Selection is not a pure score maximization step. It also weighs implementation complexity, domain specificity, and the risk of regression. The separation between proposal and selection is important because SIM conditions both stages.
The two transition types impose different forms of continuity. Cross-dataset CRSI keeps the policy while changing the task collection; cross-model CRSI keeps the benchmark and split while changing the policy:
$$\mathcal{X}_S\neq\mathcal{X}_T,\;M_S=M_T\qquad\text{or}\qquad \mathcal{X}_S=\mathcal{X}_T,\;M_S\neq M_T.$$
Both transitions require a shared executable harness interface. The reported target improvement is the final held-out score minus the starting score under the target policy and benchmark. It is not the gain obtained during source evolution.
Inside Self-Improvement Memory
Entries are indexed by failure hypotheses
SIM represents a recurring harness problem as a root-cause hypothesis and a sequence of attempts. Each entry has the form
$$m_i=\bigl(c_i,[(a_{ij},o_{ij},p_{ij})]_{j}\bigr).$$
For attempt $j$, $c_i$ is the root cause, $a_{ij}$ describes the modification and its rationale, $o_{ij}$ contains predicted and observed outcomes, and $p_{ij}$ records provenance such as domain, benchmark, policy, and round. The current memory is $\mathcal{M}_t=\{m_i\}$. This structure forces a later agent to confront the problem before examining the implementation that was supposed to solve it.
The order of evidence matters. Predictions about which tasks an edit should repair are written before evaluation and checked afterward. If an edit was expected to fix three tasks but fixed one, the two missed predictions remain visible. A flat store of successful patches would erase exactly the evidence needed to judge whether a class of intervention is dependable.
Both proposal and selection read the same history
With SIM in the loop, the optimization equations become
$$\mathcal{C}_t=P(H_t,F_t,D_t;\mathcal{M}_t),\qquad H_{t+1}=\mathcal{S}(H_t,F_t,\mathcal{C}_t,F_t^{\mathcal{C}};\mathcal{M}_t).$$
The paper uses no retrieval layer and no truncation in the main method. Every candidate is proposed with the full current SIM in the proposer's prompt, and the selector reads the same record. This gives the agents access to prior diagnoses, attempted fixes, failed predictions, and the evaluation context in which those results were obtained.
The proposer must either cite an existing root cause or declare that it is opening a new one. The selector can use the record to recognize a repeated dead end, distinguish a domain-specific fix from a general requirement, and estimate regression risk. The record therefore changes not only what gets proposed but what gets accepted.
The appendix adds behavioral guidance. Treat memory as prior experience rather than a fixed rule. Check the domain, benchmark, and model behind an entry. Start from an old diagnosis when it explains the current failure, but revise the implementation when the setting changes. Revisit a rejected direction only when new evidence justifies it. Give task-level evidence more weight in similar settings and less weight when both model and benchmark change.
Only accepted edits update memory, including unsuccessful ones
At the end of a round, the selected edit $e_t$ is appended to the memory entry it cites:
$$\mathcal{M}_{t+1}=U(\mathcal{M}_t,e_t).$$
An edit is recorded even when it regresses or fails to repair a predicted task. If no proposal is accepted, the update is an identity operation:
$$U(\mathcal{M}_t,\varnothing)=\mathcal{M}_t.$$
Entries with identical root-cause text are merged without discarding their attempt histories. This is a memory of evidence, not a commitment to keep every old implementation. The final harness may contain a revised version, while SIM still preserves the earlier rationale and outcome that led to the revision.
The four transfer conditions isolate memory
Cold Start begins with the seed harness and no SIM. Harness Transfer begins with the final source harness and no SIM. Problem-Only Memory begins from the same source harness but stores only root-cause text. Full SIM begins from that same harness with the complete record. The last three conditions therefore have identical starting code.
Each memory condition persists throughout target optimization. Full SIM appends target attempts, Problem-Only Memory adds target diagnoses, and Harness Transfer remains without SIM. All conditions retain the ordinary target-run history $D_t$. They also share the target policy, evolve and held-out tasks, ten rounds, and five candidates per round.
This design rules out a simpler explanation for the main results. A gain cannot be credited to a stronger seed, a better starting patch, or a larger target search budget because those factors are matched. The central comparison isolates the information available while continuing from the same evolved artifact.
Transfer separates a requirement from its implementation
Source optimization begins with an empty SIM and produces $(H_S,\mathcal{M}_S)$. Full SIM carries both into the target setting. The proposer and selector use each earlier edit's rationale, result, and evaluation context to decide whether that edit still applies. The current agent may retain, revise, replace, or remove a harness modification while the older evidence remains in memory.
This distinction is central to CRSI. A source patch may encode a reusable requirement through an implementation tied to one model's message format. If the requirement survives but the format changes, the correct response is to preserve the requirement and replace the implementation. A list of diffs cannot make that distinction, while a problem-centered record can.
The following diagram follows the actual paper procedure. Acceptance is not the end of a memory item because later rounds can revise the selected implementation or append new evidence.
flowchart TD
A["Source harness Hs and memory Ms"] --> B["Dataset or model transition"]
B --> C["Proposer reads Ht, Ft, Dt, and Mt"]
C --> D["Five candidate harnesses"]
D --> E["Selector screens and ranks candidates"]
E --> F["Execute retained candidates on evolve set"]
F --> G{"Candidate accepted"}
G -->|yes| H["Append root cause, edit, prediction, outcome, provenance"]
G -->|no| I["Keep harness and memory unchanged"]
H --> J["Next target round"]
I --> J
J --> K{"Ten rounds complete"}
K -->|no| C
K -->|yes| L["Evaluate final harness on held-out set"]
Experimental Protocol
The study spans coding, text classification, agentic workspace tasks, and engineering design. Coding moves from Terminal-Bench 2.1 to Terminal-Bench 4. Text classification moves from a three-dataset source suite to eight target datasets that do not appear in source optimization. Workspace tasks move from Harvey LAB to an equal-weight aggregate of JobBench, GDPval, and APEX-Agents. Engineering moves from EngDesign to Frontier-Eng.
Gemini 3.8 Flash is the task policy in both stages of the cross-dataset comparisons and also acts as proposer and selector. In the Opus-to-Gemini experiment, Claude Opus 5.5 solves the source tasks and evolves the harness, after which Gemini 3.8 Flash continues on the same benchmark and split. Policy weights remain frozen. Every source or target RSI run uses ten rounds, five candidate harnesses per round, and five task rollouts per evaluation task.
Coding uses pass@1 over the five rollouts:
$$\mathrm{pass@1}=\frac{100}{n}\sum_{i=1}^{n}\frac{1}{5}\sum_{r=1}^{5}y_{ir}.$$
The binary variable $y_{ir}$ records whether rollout $r$ of task $i$ passes the task verifier. Infrastructure failures remain in the denominator and receive zero. Text classification uses a pooled accuracy with equal weight per example, so a dataset contributes in proportion to its number of examples.
Results
Full SIM wins every cross-dataset transition
Across the six cross-dataset settings, Full SIM exceeds Harness Transfer by 2.43 to 4.35 points and finishes above Problem-Only Memory in every case. The largest margin over Harness Transfer appears in engineering design, where Full SIM reaches 46.96 from 39.13 while Harness Transfer reaches 42.61. The FewShotAll result also shows why source performance is not enough: the evolved source harness starts at 72.92 on the Gemini target, below the 75.72 seed, yet Full SIM continues to 77.92.
| Transition | Cold Start | Harness Transfer | Problem-Only | Full SIM | Gain vs. transfer |
|---|---|---|---|---|---|
| Coding / AHE | 19.39 | 20.61 | 18.79 | 24.85 | +4.24 |
| Coding / Terminus-2 | 19.39 | 19.39 | 20.61 | 21.82 | +2.43 |
| Text classification / NoMemory | 72.00 | 71.00 | 73.40 | 73.60 | +2.60 |
| Text classification / FewShotAll | 75.92 | 74.12 | 75.52 | 77.92 | +3.80 |
| Workspace / ReAct | 82.98 | 85.07 | 85.54 | 87.89 | +2.82 |
| Engineering / ReAct | 40.87 | 42.61 | 42.61 | 46.96 | +4.35 |
Table 1 reports final held-out scores and subtracts Harness Transfer from Full SIM. It corresponds to Appendix Table 5.
Problem-Only Memory is inconsistent. Root-cause text can identify where a previous edit was aimed, but it omits which change was made, what the change predicted, and what actually followed. In some settings this diagnosis-only record falls below Harness Transfer. The full record connects the abstraction to testable interventions, which is the stronger explanation for its consistent lead.
Source gains also do not guarantee a better target initialization. Source held-out scores improve by 2.75 to 6.41 points in all six settings, yet the FewShotAll source harness is worse than the target seed. The source artifact has learned something specific to the source data or interface. CRSI does not assume that transfer is automatically helpful; it uses target feedback plus source history to decide which parts remain relevant.
Cross-model transfer rewards reinterpretation
The Opus 5.5 harness is initially worse than Gemini 3.8 Flash's seed in every domain, with deficits from 0.60 to 5.48 points. After target optimization, harness-only transfer remains below Cold Start in three domains. Full SIM leads all four domains and beats harness-only continuation by 3.60 to 11.33 points.
| Target domain | Gemini cold start | Opus harness, no SIM | Problem-only | Full SIM | Gain over no SIM |
|---|---|---|---|---|---|
| Coding / AHE | 19.39 | 18.18 | 20.61 | 21.82 | +3.64 |
| Text classification / FewShotAll | 82.72 | 80.20 | 81.32 | 83.80 | +3.60 |
| Workspace / ReAct | 82.98 | 73.42 | 79.22 | 84.75 | +11.33 |
| Engineering / ReAct | 40.87 | 42.61 | 46.96 | 50.43 | +7.82 |
Table 2 compares each continuation with the no-memory run initialized from the same Opus harness. Workspace gives the clearest example: both transfer conditions start at 68.41, but harness-only continuation finishes at 73.42 while Full SIM reaches 84.75. The common start turns the target continuation into the relevant explanatory variable.
Continuation across model generations is useful but not monotonic. On Gemini 2.5 Flash, 3.5 Flash, 3.8 Flash, and 4 Argon, Fresh RSI scores are 21.4, 73.3, 88.9, and 95.6, while CRSI scores are 22.4, 76.8, 90.5, and 94.8. CRSI gains 3.5 and 1.6 points in the middle two transitions but loses 0.8 on Gemini 4 Argon. Historical evidence can expire when the policy changes enough.
A separate shared-harness study pools feedback from Grok 4.20 Reasoning, Claude Opus 5.5, and Gemini 3.8 Flash. Unified synthesis reaches 75.24 on held-out examples, effectively matching the 75.20 average of separately evolved harnesses and exceeding the 73.28 shared seed. This is a descriptive comparison of one common artifact, not a substitute for the matched-start memory experiment.
Nine of ten independent coding continuations improve
The coding study repeats source optimization ten times from Terminus-2 on TB2.1. Each independently evolved harness initializes two matched TB4 runs, one with SIM and one without. From a shared average starting score of 18.36, no-memory continuation finishes at 19.33 and Full SIM at 21.70. The test unit is the independent source run, not an individual rollout.
| Run | TB4 start | No SIM | Full SIM | Difference |
|---|---|---|---|---|
| 1 | 18.18 | 20.61 | 22.42 | +1.81 |
| 2 | 16.97 | 17.58 | 19.39 | +1.81 |
| 3 | 17.58 | 20.00 | 20.61 | +0.61 |
| 4 | 18.18 | 18.18 | 23.64 | +5.46 |
| 5 | 17.58 | 18.79 | 20.61 | +1.82 |
| 6 | 19.39 | 20.00 | 23.64 | +3.64 |
| 7 | 18.79 | 20.00 | 21.82 | +1.82 |
| 8 | 18.18 | 20.00 | 19.39 | -0.61 |
| 9 | 19.39 | 18.18 | 23.03 | +4.85 |
| 10 | 19.39 | 20.00 | 22.42 | +2.42 |
Table 3 matches Table 3 of the paper. SIM wins nine pairs, with a mean paired advantage of 2.36 points and a 95% confidence interval of $[1.05,3.68]$. A two-sided paired test gives $t(9)=4.06$, $p=0.003$, a signed-rank check gives $p=0.006$, and the standardized paired effect is $d_z=1.28$. The sole regression is 0.61 points.
The retained-harness curves agree. Full SIM is best or tied in every target round for both coding seeds. From the AHE seed, it passes the strongest control's final score of 20.61 by round 5; from Terminus-2, it does so by round 2. The gain is not a last-round spike caused by a single lucky candidate.
Source retention is measured in the AHE transition. Full SIM, Problem-Only Memory, and Harness Transfer all begin at 90.22 on held-out TB2.1. Full SIM ends at 88.89, Problem-Only Memory at 88.44, and Harness Transfer at 87.56. Out of 225 rollouts, Full SIM loses three successes, Problem-Only Memory loses four, and Harness Transfer loses six. Full SIM also gains 6.67 points on TB4, compared with 0.61 for Problem-Only Memory. The record supports adaptation while reducing forgetting, but it does not make forgetting disappear.
Mechanistic Evidence From a Completion Gate
The clearest case study follows a completion-control interlock from TB2.1 to TB4. Source RSI adds it over rounds 6, 8, and 10. At that point it rejects completion after parse failures, commands that did not execute, truncated or discarded command blocks, terminal crashes, assertion failures, HTTP 404 responses, and test errors. The implementation grows to 275 lines and raises successful rollouts from 1/5 to 5/5 on configure-git-webserver and circuit-fibsqrt.
The target task changes the notion of sufficient evidence. An agent routes nine photonic waveguides, writes a feasible layout, and sees its constraint checker pass. The score is a route cost, so feasibility does not imply optimality. None of the source rejection paths fires, and the harness allows the episode to end. The new question is whether the feasible solution is good enough, not whether a command actually ran.
Full SIM retains the inherited gate and uses its rationale and outcomes to add target checks for feasibility versus optimality. The no-memory run sees the same code but treats it without the source history. It tightens the existing error patterns, repairs completion deadlocks twice, and attributes the inherited gate to its own first target round. The abstract requirement survives in both conditions; the record is what helps the target agent revise the implementation rather than merely intensify it.
The case also exposes the limits of a record. Two target edits for the optimality problem are accepted, but neither fixes the predicted tasks. SIM preserves those missed predictions instead of presenting every accepted change as a success. This distinction is essential for later rounds: acceptance means the selector preferred an intervention under current evidence, not that the intervention was causally effective.
Limitations and Open Risks
The authors state that independent repeated RSI runs are limited to coding. Each condition in the other domains is a single complete run, so those scores are descriptive rather than estimates of optimization-run variance. The large workspace gap is informative, but it does not provide the same across-run interval as the ten-pair coding study.
The experiments cover settings with compatible harness interfaces. If a model transition changes tool protocols, message schemas, or the execution environment, historical reasoning may be only partly transferable. The optimizer's planning and search procedures are also held fixed, and SIM is evaluated as a complete procedure without component ablations for root causes, edits, outcomes, and provenance.
Compute cost is another practical constraint. One coding round takes about four hours, including roughly 1.7 hours for evaluation and 34 minutes for candidate sampling. A run has ten rounds, and the ten-pair study repeats source and target optimization many times. The methods match rounds and candidate counts but not token use or wall-clock cost, so deployment must budget for the extra memory context.
The main method sends all SIM entries to both proposer and selector without retrieval or truncation. This is reasonable for the observed run lengths and preserves complete evidence, but it is not a long-horizon solution. A much longer stream will eventually require summaries, relevance filtering, invalidation, or hierarchical storage. Naively appending every attempt can make stale evidence expensive and distracting.
Workspace scores use Gemini 3.8 Flash as the judge across benchmarks. Fixing the judge prevents conditions from seeing different evaluators, but it does not remove model-specific preferences or systematic grading error. The paper also has no public implementation for v1, so reproduction depends on reconstructing prompts and wrappers from the paper and appendices.
A broader design question remains: how should memory distinguish a failed edit from a failed prediction? The paper records both, which is better than a binary accepted/rejected flag, but it does not fully model interactions among edits. Two individually ineffective changes may work together, and an effective change may depend on an earlier mechanism. Future work could add component ablations and longer transfer chains to separate these effects.
Conclusion
The paper reframes continual harness learning as a problem of transferring improvement experience rather than retaining only the latest artifact. CRSI resumes from an evolved harness when the benchmark or policy changes. SIM carries root causes, selected modifications, rationales, predictions, outcomes, and provenance into the next setting, then appends new target evidence. Full SIM leads all six cross-dataset and all four cross-model main comparisons, and nine of ten independent coding continuations benefit.
The most transferable result is conceptual. A harness diff records what the code became, while memory records why that code exists, where it was tested, and when it should be revised. The completion-gate case shows that a rule can remain correct while its implementation becomes insufficient. CRSI is useful because it can preserve that distinction, turning cross-setting optimization into evidence-guided revision rather than mechanical inheritance.
What survives a model or benchmark transition should be the causal trail from problem to edit to outcome, not merely the last patch.
SOURCE LINKS



