PAPER DEEP DIVE
LongHarness: Stateful Evidence Management for Agentic Reasoning over 100M-Token Corpora
Corpus-level long-context reasoning requires agents to acquire and integrate evidence across document collections that exceed their context windows. Existing agent harnesses enable incremental access to these corpora, but acquired evidence and evolving conclusions often remain entangled in a growing interaction history. This implicit state management can lead to two coupled failure modes: unbounded exploration, where retrieval expands the interaction history without resolving decision-relevant information gaps, and evidence-state drift, where previously acquired evidence or subsequent corrections fail to inform the current conclusion. We present LongHarness, a model-agnostic harness that makes the interaction between acquired evidence and evolving conclusions explicit. Its core mechanism, Evidence-Conclusion Coevolution, uses unresolved conclusions to guide exploration toward missing information, while newly acquired evidence strengthens or revises the conclusions that directed the search. LongHarness couples this process with Evidence Lifecycle Management, which keeps accumulated evidence current and traceable throughout the interaction. Together, these mechanisms focus retrieval on remaining information needs and keep conclusions aligned with current evidence. We evaluate LongHarness on OfficeQA Pro V1 and V2, AA-LCR, and LongBench v2 using GLM-5.3-Flash and Qwen3.8-Flash. Compared with direct reasoning using the same backbone, LongHarness improves GLM-5.3-Flash exact match on OfficeQA Pro from 15.8% to 60.2% on V1 and from 4.4% to 61.1% on V2. Against state-of-the-art agent harnesses, it surpasses Claude Code by 12.0 percentage points on V1 and Codex by 18.9 points on V2. With Qwen3.8-Flash, LongHarness reaches 61.7% on V1, outperforming Claude Code by 8.3 points and Codex by 12.0 points.
Original title: LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management
Authors: Songtao Huang, Zhijie Zhong, Xiaohan He, Shuaiyu Zhang, Zhiyin Yu, Li Kang, Heng Zhou, Yutao Fan, Xiangchao Yan, Bo Zhang, Shiyang Feng
Affiliations: Fudan University, Shanghai Artificial Intelligence Laboratory, Peking University, Shanghai Jiao Tong University, University of Science and Technology of China, Harbin Institute of Technology
Paper: https://arxiv.org/abs/2610.13075
Code status: The paper body, appendices, and references do not point to an author-provided implementation repository, and the public listing fields are empty, so this work does not ship public code. The reusable public resource is the AA-LCR-1.1 evaluation set that the paper cites (https://huggingface.co/datasets/ArtificialAnalysis/AA-LCR), together with the official OfficeQA Pro and LongBench v2 benchmarks. Those are datasets, not the method implementation.
One-Sentence Summary
LongHarness separates what an agent has read and what it currently concludes into two explicit, mutually driving persistent states: unresolved gaps in the conclusion decide what to retrieve next, and newly admitted evidence strengthens or revises the conclusion. With this loop, GLM-5.3-Flash raises exact match on OfficeQA Pro from 15.8% to 60.2% on V1 and from 4.4% to 61.1% on V2, beating Claude Code by 12.0 points on V1 and Codex by 18.9 points on V2.
Background and Motivation
Real document questions rarely depend on one or two passages. The supporting facts are scattered across many files, versions, and tables, and the collection as a whole is far larger than a model's context window. The paper calls this setting corpus-level long-context reasoning: an agent has to discover the material, accumulate it across multiple interactions, and only then turn it into an answer. OfficeQA Pro V1 makes the scale concrete with 133 questions over 697 U.S. Treasury documents, roughly one hundred million tokens, while V2 expands the corpus to 1,435 documents and adds composite questions that require values from multiple sources.
The intuitive fix is a longer window. Recent long-context models can indeed take in more material in a single call, but the paper argues that this does not remove the difficulty. Even when the relevant content fits, models stay sensitive to where the evidence sits and to distracting passages. Length expands what is available in one invocation, but it does not settle the harder question of which material should actually be used. Once the corpus exceeds the window, a single call cannot hold the candidates at all, and an external retrieval loop becomes necessary to pull material back piece by piece.
That is why the field turned to agent harnesses that let the model decide what to search, which passage to read, and what to remember. These harnesses do give models adaptive access to very large collections and some form of persistent storage. The paper's criticism is sharper than a generic complaint about memory: existing harnesses entangle the evidence that was retrieved with the conclusions that are still evolving inside one growing interaction history. The conclusion is buried between reasoning steps, tool calls, and observations, is not an explicit state, and has no stable reference structure.
Implicit state management produces two coupled failure modes. The first is unbounded exploration: retrieval keeps happening without resolving the information gaps that actually matter to the decision, the history grows, and the interaction budget runs out. The second is evidence-state drift: previously acquired evidence, or a later correction, gradually disconnects from subsequent reasoning, so the final decision rests on stale or incomplete evidence. The two feed each other. Drift hides which information needs have already been resolved, which prompts more retrieval, which lengthens the history and makes the relevant evidence even harder to recover.
The paper also offers a clean lens that separates the whole process into three levels. Level one is evidence retrieval, locating relevant passages while filtering distractors. Level two is evidence editing, selecting, reconciling, and revising retrieved content. Level three is evidence transformation, turning the resulting evidence into an answer. Existing training methods supervise retrieval trajectories and memory edits, and existing harnesses provide adaptive access and persistent storage, yet the transition from accumulated evidence to an evolving conclusion stays implicit. The conclusions that determine what to search for next and what answer to produce remain embedded in session history, free-form notes, or transient reasoning. LongHarness is aimed squarely at that gap.
Preliminaries and Problem Formulation
The task is written as follows: given a question $q$ and a set of documents $D=\{d_1,\ldots,d_n\}$ that live outside the prompt, an agent acquires evidence incrementally through an external environment under a resource budget. Its state has three parts: source-linked evidence that has been acquired, evidence-linked conclusions and unresolved gaps, and the prior reasoning, actions, and observations. The paper writes $$S_t=(E_t,C_t,H_t),$$ where $E_t$ is the active evidence, $C_t$ is the conclusion state, and $H_t$ is the interaction history. Treating both evidence and conclusions as first-class state, rather than letting them dissolve into history, is the starting point of the whole method.
On each turn the policy reads the question, the document map, the current state, and the remaining budget, then chooses an action and optionally updates the conclusion: $$(a_t,\widehat{C}_t)\sim\pi_{\theta}(\cdot\mid q,M_D,S_t,B_t).$$ The environment then executes the action and returns the next environment state and observation: $$(x_{t+1},o_{t+1})\sim\mathcal{P}_{D}(\cdot,\cdot\mid x_t,a_t).$$ The action space covers search, bounded reading, EvidenceBank operations, computation, and final submission. Before any update is exposed to later turns, the controller validates the action structure, checks that source references are registered, and enforces the budget.
To make the dependency between acquisition and revision precise, the paper defines two update operators for one reasoning cycle, and they recur throughout the method. The first describes exploration guided by the current conclusions together with admission of the resulting statements into the evidence bank; the second describes updating the conclusion state from the active evidence. A single agent policy realizes both operators over one or more interactions. The equations fix the direction of dependency while leaving exploration and revision free to interleave adaptively.
Method in Detail
Overview: Evidence and Conclusions as Coupled Persistent State
LongHarness is a model-agnostic runtime harness. It does not train a new policy. Instead it maintains an EvidenceBank and a Conclusion State as runtime state that persists across turns, and lets the two coevolve inside one shared action space. The EvidenceBank stores source-linked statements extracted from passages the agent has read. The Conclusion State records the agent's current interpretation of that evidence: established claims, supported hypotheses, and unresolved gaps. Keeping them apart matters because evidence is the record of what the documents say, while a conclusion is the inference that turns evidence into a decision; explicit references connect the two instead of blending them.
Each unresolved gap names information needed to establish, qualify, or revise the current conclusion. A conclusion is therefore not a static string but a workbench with a gap list: the gaps say what to look for in the corpus next, and returning evidence decides whether the conclusion must change. The overview figure shows a single agent moving among the EvidenceBank, the Conclusion State, and the external documents.
Figure 2: Overview of LongHarness. A single agent maintains an EvidenceBank and a Conclusion State while interacting with external documents. Evidence-Conclusion Coevolution uses unresolved conclusions to guide exploration and accepted evidence to revise those conclusions, while Evidence Lifecycle Management governs which statements remain active and preserves their source references throughout the interaction.
Mechanism One: Evidence-Conclusion Coevolution
The separation of evidence from inference is operationalized by two operators. Within a reasoning cycle, the current conclusion state controls exploration and admits the newly retrieved statements into the evidence bank: $$E^{+}=\mathcal{A}_{\theta}(q,D,E,C),$$ and then the active evidence in the bank updates the conclusion: $$C^{+}=\mathcal{R}_{\theta}(q,E^{+},C).$$ The first turns "what should we find" into executable search and reading; the second turns "what did we find" into a strengthening, qualification, or rewrite of the conclusion. The equations pin down the direction of dependency without dictating how many steps each side takes, so exploration and revision can interleave as the situation demands.
On the conclusion-guided exploration side, the Conclusion State decides what to search for and which passage to read next. For each claim the agent first identifies which observations would establish, qualify, or refute it, then uses those requirements to form queries and choose reading scopes. Competing hypotheses motivate searches that distinguish between them, and unresolved gaps pull attention toward missing premises. When an observation leaves a gap open, the agent may reformulate the query or widen the reading scope, but exploration stays tied to the information needed to advance the conclusion instead of expanding the history for its own sake.
On the evidence-grounded revision side, after observing the result of an EvidenceBank update the agent reassesses each claim in the Conclusion State against the active evidence, deciding whether it is still supported, needs qualification, or must be revised. It updates the affected claims together with their evidence references and unresolved gaps, keeping the Conclusion State aligned with current evidence. Before final submission it also folds in any relevant evidence not yet reflected in the state and checks whether the remaining gaps could change the requested decision. This step is the direct answer to evidence-state drift: once a correction is admitted, it has a path into the final computation.
The diagram below is drawn straight from the paper's action space. A conclusion gap drives search and read; text read registers source references that an upsert admits as evidence; after the update is observed, revise corrects or withdraws entries; the conclusion is then revised; once the gaps close, compute and final take over.
flowchart TD
Q["Question q and external documents D"] --> S["Persistent state S_t"]
S --> G{"Any unresolved gaps in the conclusion"}
G -->|gaps remain| SR["search locate candidate documents"]
SR --> RD["read bounded reading and register source refs"]
RD --> EB["EvidenceBank upsert admit source-linked statements"]
EB --> OBS["Observe the update result"]
OBS --> REV["EvidenceBank revise correct or retract entries"]
REV --> CO["Revise conclusions update claims and gaps"]
CO --> G
G -->|gaps closed| CP["compute evaluate expression over cited evidence"]
CP --> F["final submit answer with active evidence refs"]
Mechanism Two: Evidence Lifecycle Management
The second mechanism governs where evidence comes from, how it persists, and when it leaves. Evidence Lifecycle Management controls how source-linked statements enter, remain in, and exit the active evidence state. A statement starts as a candidate extracted from a passage the agent read, becomes active once admitted, and may later be revised or retired as new evidence arrives. The lifecycle has three requirements: traceability back to observed source passages, persistent availability across interactions, and explicit updates when the evidence supporting a statement changes.
The EvidenceBank meets all three. For traceability, every candidate must cite source references registered by earlier read actions; the controller checks reference validity while the agent judges whether the statement faithfully represents its sources. For persistence, admitted statements remain available in later interactions so the agent need not reconstruct evidence from earlier observations. For explicit revision, a corrected statement is admitted as a new active entry and the previous one is marked superseded, while a retraction removes the previous entry without a replacement. The agent observes the EvidenceBank update before revising affected conclusions, and final references resolve to active statements and their underlying source passages.
The implementation exposes only two explicit operations. An upsert admits a batch of source-linked statements, and a revise replaces one active entry with a replacement statement or withdraws it with a null replacement. Each proposal is validated independently against references registered by prior reads, so a batch may be partially accepted. Accepted entries keep immutable identifiers, and corrections are recorded through revise rather than in-place edits. Superseded or retracted entries stay in the revision record but cannot be cited in compute or final requests. The paper is explicit about one consequence: when evidence becomes inactive the transition log flags affected conclusion references, but the runtime does not automatically delete or rewrite those claims, a point revisited in the limitations.
The Agent-Environment Interaction Interface
A shared action interface lets the policy interleave acquisition, maintenance, and use inside one turn sequence. search locates documents or passages relevant to an information need and returns candidate locations with navigation snippets. read inspects selected locations under a bounded extent and returns source text with registered source references. EvidenceBank commits or revises source-linked statements. compute evaluates a deterministic numerical expression over evidence values and validates the declared evidence references. final submits the answer with supporting references. The interface deliberately separates candidate discovery from source inspection: a search snippet is only a navigation aid, and only a reference returned by read is eligible to support an EvidenceBank statement.
Each turn the policy picks exactly one operation, and the controller validates its structure, source references, and resource requirements before execution; an invalid action returns an error observation without modifying persistent state. EvidenceBank mutations defer conclusion updates until the agent observes the result, after which the policy revises affected claims and references through optional partial updates whose omitted fields are retained. The paper fixes the resource envelope at 200 turns, at most 60 search calls, a cumulative read budget of one million tokens, and at most 16,384 tokens per read. Historical observations are retained in full rather than replaced by summaries or placeholders, which is exactly why a persistent request failure, including rejection of an oversized request, terminates the episode.
Read together, the interface and the state are surprisingly economical. LongHarness does not add a sprawling multi-agent system and does not touch the model weights. It adds two structured states and a set of controlled operations at runtime. What does the work is making explicit the part of the context that decides what to search for next and what to answer, and maintaining it every time the evidence changes. The ablations make the point concrete: remove every persistent state and keep only the interaction history, and accuracy immediately drops.
Experiments
Evaluation Setup
The paper evaluates on four settings: OfficeQA Pro V1 and V2, AA-LCR-1.1, and the Long subset of LongBench v2, spanning external-corpus retrieval and reasoning over provided contexts. OfficeQA Pro V1 has 133 questions over 697 documents at roughly one hundred million tokens; V2 grows to 90 questions and 1,435 documents and emphasizes composite questions that combine values from different series, versions, or documents. AA-LCR-1.1 has 100 questions associated with 30 document sets of 234 documents in total, averaging about 99,325 tokens per set, and its questions call for synthesis, temporal reasoning, conditional logic, or numerical computation across documents. The LongBench v2 Long subset has 108 questions whose contexts range from about 132,000 to 1.38 million words with a median near 204,000 words.
The main comparison fixes the backbone to GLM-5.3-Flash and Qwen3.8-Flash and compares Codex CLI 0.135.0, Claude Code 2.1.216, and RLM. All harnesses receive the same questions and corpus, with no access to reference answers or gold source annotations. Codex and Claude Code each allow up to 200 model turns and 200 tool calls, RLM allows 30 iterations, and LongHarness allows up to 200 steps under a one-million-token cumulative read budget and a limit of 60 search calls. One honest caveat belongs here: because of computational constraints, the main comparison runs each harness once per setting, and the three-run study in the appendix covers only GLM-5.3-Flash.
Main Results: A Clear Lead on Corpus-Level Retrieval
Even when given BM25-retrieved passages in a 256K window, direct-reasoning baselines reach at most 23.2% exact match on OfficeQA Pro V1 and 5.6% on V2. The paper's explanation is that at corpus scale a fixed window cannot hold all potentially relevant evidence, and passages chosen before reasoning may omit information needed to resolve intermediate conclusions. LongHarness uses those conclusions to guide further retrieval and accumulates the resulting evidence across interactions, lifting GLM-5.3-Flash exact match from 15.8% to 60.2% on V1 and from 4.4% to 61.1% on V2.
Against the strongest competing harness, LongHarness is ahead by 12.0 points on V1 and 18.9 points on V2. The larger V2 margin fits an important observation: when more scattered values must be located, reconciled, and combined, carrying evidence across interactions pays off more. With Qwen3.8-Flash, LongHarness still leads on both versions, reaching 61.7% on V1 and 51.1% on V2; its advantage over the strongest competing harness is 8.3 points on V1 but narrows to 1.1 points over Codex on V2. The paper does not hide this, stating plainly that the size of the gain varies with the backbone and setting. On AA-LCR it scores 84.0% and 87.0%, on LongBench v2 Long 75.9% and 76.9%, and its average Exact score across the four settings reaches 70.3% and 69.2%.
| Backbone | Harness | OfficeQA V1 Ex. | OfficeQA V2 Ex. | AA-LCR | LongBench v2 | Avg. |
|---|---|---|---|---|---|---|
| GLM-5.3-Flash | Direct | 15.8 | 4.4 | 80.0 | 62.0 | 40.6 |
| GLM-5.3-Flash | Codex | 46.6 | 42.2 | 77.0 | 69.4 | 58.8 |
| GLM-5.3-Flash | Claude Code | 48.1 | 35.6 | 79.0 | 63.0 | 56.4 |
| GLM-5.3-Flash | RLM | 45.1 | 38.9 | 56.0 | 60.2 | 50.0 |
| GLM-5.3-Flash | LongHarness | 60.2 | 61.1 | 84.0 | 75.9 | 70.3 |
| Qwen3.8-Flash | Direct | 15.8 | 4.4 | 80.0 | 54.3 | 38.6 |
| Qwen3.8-Flash | Codex | 49.6 | 50.0 | 81.0 | 69.4 | 62.5 |
| Qwen3.8-Flash | Claude Code | 53.4 | 41.1 | 83.0 | 63.9 | 60.3 |
| Qwen3.8-Flash | LongHarness | 61.7 | 51.1 | 87.0 | 76.9 | 69.2 |
Table 1: Main results across four benchmarks. OfficeQA Pro reports exact match and accuracy within 5% relative error, while AA-LCR and LongBench v2 report Exact accuracy; only the Exact columns and the averages are excerpted here. Values are taken from Table 1 of the paper.
Ablations: Both Mechanisms Earn Their Keep
Five ablations ask whether evidence-conclusion coevolution and evidence lifecycle management are truly critical. They remove all persistent states while keeping the interaction history; remove evidence-conclusion coevolution while keeping the EvidenceBank but not the Conclusion State or its retrieval feedback; restrict evidence to naive append-only accumulation; keep both states but remove the explicit instruction to guide retrieval with conclusion gaps; and replace structured conclusions with editable free-text notes. The table shows that replacing structured conclusions with free-text notes costs the most, 6.8 points on V1 and 7.8 on V2, while append-only accumulation costs 10.0 points on V2, evidence that revising and retiring evidence matters.
| Variant | OfficeQA V1 Ex. | V1 drop | OfficeQA V2 Ex. | V2 drop |
|---|---|---|---|---|
| LongHarness (full) | 60.2 | - | 61.1 | - |
| w/o persistent states | 55.6 | 4.5 | 53.3 | 7.8 |
| w/o evidence-conclusion coevolution | 54.9 | 5.3 | 51.7 | 9.4 |
| Naive evidence accumulation | 56.4 | 3.8 | 51.1 | 10.0 |
| w/o explicit conclusion-guided exploration | 55.6 | 4.5 | 53.3 | 7.8 |
| Generic working notes | 53.4 | 6.8 | 53.3 | 7.8 |
Table 2: Ablations with GLM-5.3-Flash on OfficeQA Pro. Ex. is exact-match accuracy (%) and the drop is the point difference from full LongHarness, computed before rounding. Values are taken from Table 2 of the paper. Every ablation loses more on V2, consistent with the greater importance of maintaining and reconciling evidence when questions combine values across sources.
Trajectory Analysis: Where the Failures Live
The paper audits incorrect episodes with GPT-5.6-Luna at temperature 0 and splits failure into three stages: whether retrieval finds the required evidence, whether editing retains and adopts it in the final decision, and whether transformation converts adopted evidence into an exactly correct answer. Writing $E_1$ for the presence of required evidence in the trajectory, $E_2$ for its retention and reconciliation into the final decision, and $E_3$ for an exactly correct answer, the stage rates are $$r_1=P(E_1),\qquad r_2=P(E_2\mid E_1),\qquad r_3=P(E_3\mid E_1,E_2),$$ with editing and transformation rates computed only over cases that pass the preceding stage.
LongHarness matches Claude Code's retrieval coverage on V1 and achieves higher conditional evidence-adoption and answer-conversion rates. On V2 its retrieval, adoption, and conversion rates are 87.8%, 87.3%, and 79.7% respectively, and the two conditional rates stay close to the V1 values of 87.6% and 80.8%. This stability lines up with the evidence-state-drift diagnosis: drift does not hurt retrieval breadth, it hurts whether the thing that was retrieved finds its way into the final decision.
| Evidence level | V1 Claude Code | V1 Codex | V1 LongHarness | V2 Claude Code | V2 Codex | V2 LongHarness |
|---|---|---|---|---|---|---|
| Retrieval r1 | 85.0 | 72.2 | 85.0 | 72.2 | 76.7 | 87.8 |
| Evidence editing r2 | 85.8 | 83.3 | 87.6 | 84.6 | 78.3 | 87.3 |
| Transformation r3 | 66.0 | 77.5 | 80.8 | 58.2 | 70.4 | 79.7 |
Table 3: Success rates (%) for retrieval, evidence editing, and transformation with GLM-5.3-Flash on OfficeQA Pro. Values are taken from Table 3 of the paper. Higher retrieval coverage means the required evidence is more often found; higher conditional editing and transformation rates mean it is more often retained and converted into a correct answer.
Interaction length tells a related story. From V1 to V2 the median number of turns rises from 22 to 30 for LongHarness, from 21 to 116 for Claude Code, and from 19 to 43 for Codex. Codex exhausts its recorded model-turn budget on 3.8% of V1 questions and 21.1% of V2 questions, and Claude Code's distribution concentrates more heavily near 200 turns on V2. The paper's explanation is that V2 combines a larger corpus with questions that need values from multiple sources; because LongHarness records unresolved needs in the Conclusion State and keeps acquired evidence in the EvidenceBank, later retrieval can focus on what is missing and reuse values already established, rather than searching for or rebuilding the same facts. That is a plausible account of why its turn count grows so much less.
Case Study: A Correction That Was Seen but Not Used
Figure 4 of the paper walks through computing the geometric mean of 80 monthly Treasury expenditures. Codex locates the relevant table within 22 command calls but leaves missing values implicit, then performs 103 consecutive backward reads, a textbook case of unbounded exploration. It computes 4958.09 using the incorrect 4100 for May 1942, then later recognizes the corrected value 4400 but keeps the earlier result in its final answer.
Figure 4: Evidence lifecycle comparison with GLM-5.3-Flash on OfficeQA Pro V1. For the geometric mean of 80 monthly Treasury expenditures, Codex continues searching and omits a recognized correction, while LongHarness updates its evidence and returns the reference answer of 4962.46 million dollars. Circles mark Codex's incorrect input, the correction, and its final answer.
Replacing 4100 with the corrected 4400 turns the geometric mean into the reference answer 4962.46. The failure is revealing because Codex did see the correction; it simply never connected the correction to the final computation, which is precisely evidence-state drift. LongHarness folds the correction into its evidence and revises the conclusion within 10 policy actions, returning the correct value. The paper is careful to note that although Codex misses exact match, it passes the 5% tolerance threshold with a relative error of about 0.088%, which separates the two levels of failure cleanly: retrieval and reading can both succeed while adoption fails.
Figure 1: Evidence lifecycle management for corpus-level long-context reasoning. The left panel shows an agent losing an earlier constraint and continuing to search, while LongHarness retains the evidence, uses an unresolved conclusion to guide retrieval, and revises its decision when new evidence arrives. The right panel compares exact accuracy on OfficeQA Pro V1 and V2 for direct models and agent harnesses using GLM-5.3-Flash.
The distribution of interaction turns is worth reading alongside the case. In the density plots below, LongHarness moves very little from V1 to V2, while Claude Code and Codex shift markedly to the right on V2. The paper attributes this stability to explicit conclusion gaps and evidence reuse: the more a question demands cross-source verification, the more the structured state pays off.
Figure 3: Distributions of model turns per question on OfficeQA Pro V1 and V2 with GLM-5.3-Flash. Each panel compares the two versions for one harness; curves are kernel density estimates and circles mark medians.
Limitations
The first limitation is stated by the authors themselves: because of computational resource constraints, the main comparison uses one run per harness and evaluation setting, and only the appendix reports three runs for GLM-5.3-Flash, with individual scores, means, standard deviations, and maxima. That means several main-table gaps carry no paired variance estimate. The three-run study offers partial reassurance, since LongHarness scores 60.15, 60.15, and 59.40 on OfficeQA Pro V1, a standard deviation of only 0.43, and 61.11, 58.89, and 58.89 on V2, a standard deviation of 1.28, but it does not extend to Qwen3.8-Flash or to every benchmark.
The second limitation is that the gain is not uniform. With Qwen3.8-Flash, LongHarness beats Codex by only 1.1 points on OfficeQA Pro V2, far short of the 18.9-point margin on GLM-5.3-Flash. The paper writes directly that the size of the advantage varies with the backbone and evaluation setting, which warns against extrapolating a large lead on one backbone to all models.
The third limitation is built into the runtime contract. Appendix B.3 notes that the interaction history is retained in full and is not automatically truncated or compacted, and that a request which still fails on length after the client's retry handling is recorded as model_request_failed and terminates the episode, with episodes ending without an accepted answer scored as incorrect. As history grows, the system becomes increasingly sensitive to per-request context length. The paper also flags two finer verification gaps: the runtime checks whether evidence IDs are registered and active, but compute does not establish that the literal operands match the cited source values; and when evidence is superseded or retracted, the transition log flags affected conclusion references but the runtime neither rewrites those claims nor blocks later search and reading. Finally, the evidence-pipeline audit uses GPT-5.6-Luna with conditional stage rates rather than an exhaustive causal proof, so the trajectory analysis offers a consistent explanation rather than a causal verdict.
Conclusion and Outlook
The contribution of LongHarness is not a more intricate retriever. It is a re-description of what long-context reasoning is: the construction and continual refinement of the evidence supporting a decision is the reasoning process itself. Under that view, retrieval, editing, and transformation are no longer three separate stages but the same body of evidence playing different roles over time. In method, Evidence-Conclusion Coevolution couples exploration with conclusion revision, and Evidence Lifecycle Management keeps evidence current and traceable. In implementation, two states and a set of controlled operations are enough to attach the harness to different backbones.
The experiments support the design. On roughly one hundred million tokens and on a doubled corpus, GLM-5.3-Flash reaches exact match of 60.2% and 61.1%; ablations show that persistent states, conclusion-guided retrieval, and evidence revision each contribute, and that swapping structured conclusions for free-text notes clearly degrades; trajectory analysis shows the advantage lands mainly in adoption and transformation rather than in finding more. The paper also draws its boundaries clearly: a single-run main comparison, gains that vary by backbone, context risk from uncompressed history, and a runtime that does not validate literal operands. For future work, the most direct directions are multi-seed and multi-backbone replication, history compression that keeps provenance traceable, and automatic maintenance of conclusions that depend on retired evidence.
Golden Quote
The hard part of long-context reasoning is not how much material the model can read at once, but whether the system can keep track of what it has already read, whether that material still holds, and which conclusion it actually supports.
SOURCE LINKS
