OPEN SOURCE DEEP DIVE
sapper-v2: An AI Coding Multi-Agent Harness for Authorized Security Testing
A model- and CLI-agnostic multi-agent harness for authorized security work, coordinated through a persistent evidence graph that survives crashes.
sapper-v2 is not a single coding assistant with a different system prompt. It is an open-source orchestration harness that turns several interactive agents into a long-running team. The repository currently treats offensive security competitions as its main proving ground, so the boundary comes first: it is intended only for systems the operator owns or has explicit written permission to test. The bundled disclaimer prohibits unauthorized use and leaves responsibility for deployment and conduct with the operator.
The important design question is not which model sits behind a role. It is how the system records who is doing what, what evidence supports the work, and whether a result has actually been confirmed. A model session can end, a terminal process can crash, and a particular agent can be replaced. As long as facts and intents remain in the external graph, a successor can take over. That persistent coordination mechanism, rather than a particular vendor model, is the core of the project.
A directed evidence graph of facts and intents
sapper represents facts as nodes and intents as dependency-bearing edges. A fact is a confirmed observation: a host exposes a service, a response contains a particular artifact, a credential has been recovered, or a vulnerability path is backed by specific commands and output. An intent describes work to perform. Its from field lists required facts, while its to field points to the new fact produced when the work concludes.
This separation keeps plans distinct from evidence. A scanner warning cannot simply be promoted to a confirmed vulnerability; an executor must provide the command, target, port, and response evidence. Hosts, services, credentials, vulnerabilities, loot, flags, and notes have dedicated fact types so they can be counted, deduplicated, and cross-referenced. A special goal fact terminates the graph, and the project is complete only when an authorized intent points to that goal.
Evidence has three layers: a title, a summary, and a detailed document. A reader can use the title to decide whether the record is relevant, inspect the summary when needed, and open the full document only for the exact commands and observations. This reduces the amount of raw output pushed into every context window and limits the blast radius of one long, incorrect note. The scheduler can operate on compact records while preserving richer evidence for later inspection.
| Object | Purpose | Invariant |
|---|---|---|
| Fact | A confirmed observation or artifact | Must be traceable to evidence; a scanner warning is not a confirmed vulnerability |
| Intent | A unit of work with dependencies | Becomes available only after all required facts exist |
| Conclusion | An intent pointing to a new fact or the goal | Written atomically and closed against duplicate execution |
| Typed fact | Host, service, credential, vulnerability, loot, flag, or note | Carries flat key-value data for stable queries and counts |
Frontier scheduling and atomic claims
frontier() finds intents whose dependencies are currently satisfied. An intent enters the frontier only when every referenced fact exists. In-flight tasks with a claimed owner are listed first so another role does not mistake them for unassigned work. The function does not make a strategic decision for the agent; it answers a narrower question: which jobs are ready under the graph's dependency rules?
Concurrency safety comes from file locks and atomic replacement. Each graph node is a JSON file. Critical mutations acquire an exclusive lock, then read, validate, and write while holding it. New content is written to a temporary file on the same filesystem and then atomically moved into place. Claiming an intent, confirming an intent that requires human approval, and writing a conclusion together with its new fact all happen inside that lock.
The most important transaction combines conclusion and fact creation. Before writing a result, the executor rechecks whether the intent is still open. If another concurrent executor concluded it first, the later result is rejected, so the system does not leave an orphaned fact that no intent references. This addresses a coordination failure specific to multi-process work: duplicate execution followed by evidence that has lost its owner.
Roles and a zero-strategy watchdog
Roles are not permanently tied to a particular model. A commander reads the graph, judges the state, and dispatches work. A general handles difficult targets and can organize another layer of authorized executors. A scout spends a smaller budget estimating target difficulty. Soldiers provide parallel execution. The project also developed a signalman role that collects flags scattered across executors and submits them in one place, reducing duplicate actions and missed submissions.
The watchdog deliberately avoids tactical decisions. It relaunches processes, rotates keys, handles quota failures, and closes the run at the deadline. Mixing keep-alive logic with strategic control would create a second commander inside an infrastructure script. The project keeps this boundary narrow so the upper-layer policy has one clear owner.
Roles and models can be recombined. A commander or general can run on a stronger model, while scouts and soldiers use a cheaper one. The repository calls this an economy layer: reserve expensive inference for judgment and hard work, and use lower-cost inference for high-volume execution. This is an argument about scheduling economics, not a claim that one fixed model is inherently better at authorized security work.
The reason, explore, and verify loop
The standard loop has three actions. reason reads the current graph and proposes hypotheses and intents. explore claims a specific intent, performs the work, and records valid results as facts. verify checks whether the conclusion really satisfies the objective and decides whether more evidence is required. These actions are not a one-shot pipeline; they repeatedly operate on the same graph.
An auto mode can schedule those actions continuously, and verification can be disabled to reduce model usage. That tradeoff matters: skipping verification lowers cost, but it also lets weak intermediate conclusions enter later reasoning sooner. It may fit a trusted or tightly budgeted run, but it is a poor default when correctness is the primary objective. The framework exposes an autonomy setting; it does not remove the tradeoff.
On a larger graph, the system can also request blind-spot searches and brainstorming. The former looks for missing dependencies and evidence gaps, while the latter proposes new intents from existing facts. Both still have to pass through claiming, execution, and graph write constraints before changing the final conclusion.
Competition adapter, flag ledger, and offline knowledge
The repository includes a standard-library synchronous client for the TSecBench-style challenge API. It covers challenge listing, start, hint, submission, and close operations, and reads the base URL and token from environment variables or a local configuration file. This is an explicit platform coupling point, not a generic penetration-testing interface. A full competition run checks for platform configuration at startup and will not operate in the same mode without it.
Flags have their own lifecycle. The system first records the captured value, then records whether it was submitted, rejected, or already used. That distinction between capture and score matters most at the end of a run. The commander can scan flag facts for unsubmitted values instead of reopening every terminal transcript.
The cloned repository also carries an offline knowledge base. Each entry has a title, description, methodology, commands, tags, and source. Retrieval uses token overlap and simple weighting, so it works without an external vector service and can travel to an isolated match machine. The knowledge base holds reusable technique references; the fact graph holds evidence about the current engagement.
Treating graph data as untrusted
One of the project's strongest safety choices is to mark graph content and the current intent as untrusted data. A target can return a page, log line, or filename that says “ignore the previous rules” or “run this command.” The system prompt tells executors to treat that text as an observation rather than a new instruction. If an intent asks them to leave the authorized scope, expose secrets, or disable safeguards, they are told to refuse and record the request as a fact.
This does not prove that a model will always comply, but it establishes the correct trust boundary. Attack input does not arrive only through network services; it can also be copied into shared memory. Promoting target output to a higher-priority instruction is a recurring failure mode in agent systems. sapper-v2 explicitly prohibits that promotion and anchors execution to the authorized target scope.
Deployment prerequisites and platform coupling
The graph command-line workflow is comparatively light and can run on the Python standard library. The full legion is a larger system: it expects containers and Kali tools, interactive coding-agent command lines, model-gateway credentials, a key pool, and platform-provided addresses and tokens. The quick install is a reasonable way to inspect graph behavior, but it is not evidence that every external integration is ready out of the box.
The tests cover graph integrity, typed facts, arena error mapping, concurrent claims, key rotation, configuration patches, and related behavior. This review ran 48 unit tests on the cloned commit and also checked compilation of the core modules plus shell syntax for the installation and Docker scripts. Those checks establish a baseline for that snapshot; they do not replace an end-to-end test with real targets, container networking, and production credentials.
Championship claims and inconsistent records
Several repository documents consistently report a championship in the online qualifier of the third Tencent Cloud Intelligent Attack-Defense Challenge, with a score around 80.18 after a long unattended run. That narrative is important for understanding the project's motivation, but the repository does not use one consistent set of team, flag, and token counts. A promotional number should therefore not be treated as independently verified merely because it appears in the README.
| Source | Main claim | Conflict |
|---|---|---|
| README | 80.18, champion, eight unattended hours | Reports 634 teams and 85 flags, and contrasts 1 billion tokens with 4.5 billion for the runner-up |
| Battle report | 80.18, first place, roughly 475 unattended minutes | Reports 50 scored teams, more than 36 of 63 challenges solved, and about 45 flags |
| Event log | First place at 80.18 | Records about 1.03 billion tokens and lists the runner-up near 1.68 billion, not 4.5 billion |
The defensible conclusion is narrower: the repository claims and documents a championship run at 80.18 and provides battle-report material, event records, and a score curve. The “1 billion versus 4.5 billion” efficiency advantage, the number of participating teams, and the total flag count should be read as repository claims that have not been reconciled internally. The project does not currently publish one canonical results table that resolves those differences.
Who should use it, and what it does not promise
The project is relevant to developers studying long-horizon multi-agent coordination, shared fact graphs, terminal-agent orchestration, and authorized security workflows. A single code change is usually easier with one coding assistant. A task that needs several agents to persist for hours and recover a shared evidence trail is where the graph model and atomic writes become useful.
It should not be presented as an unsupervised offensive tool. The operator still defines authorization, prepares isolation, manages model and platform credentials, checks command boundaries, and remains responsible for the final action. The accountability tracer mentioned in the repository is still planned, not a mechanism that exists in the reviewed release.
As an engineering artifact, sapper-v2 contributes less to “making one model smarter” than to giving multiple forgetful, crash-prone, and duplicate-prone agents an external coordination discipline. That power also creates risk: the more successfully target output and cross-session plans persist, the more important authorization, isolation, and audit boundaries become. Those boundaries should remain visible even when the championship story is the headline.