Agent World Model in the Wild: Computer-Use Agent Environment Synthesis

Paper Metadata

Title: Agent World Model in the Wild: Computer-Use Agent Environment Synthesis (WildAWM for short)

Authors: Zhaoyang Wang, Qianhui Wu, Haorui Wang, Simon Yu, Baolin Peng, Canwen Xu, Hao Cheng, Wenlin Yao, Boyi Liu, Yite Wang, Jianfeng Gao, Huaxiu Yao. Affiliations: UNC-Chapel Hill, Microsoft Research, Georgia Tech, Northeastern University, and independent researchers (corresponding: zhaoyang@cs.unc.edu, qianhuiwu@microsoft.com).

Links: arXiv:2610.26828 (v1, 2026-10-19, cs.AI, arXiv perpetual non-exclusive license) · full HTML on arXiv · project page

Code status: fully open-sourced, released the same day as the paper (2026-10-19) - paper, code, environments plus tasks, and trained weights. github.com/MSR-Orchard/WildAWM is about 35 MB and splits into four packages: synthesis/ (the world and task synthesis pipeline, including five Dockerfiles, a dashboard, and the worldctl service), evaluation/ (the unified awmeval harness covering eight out-of-domain benchmarks), wildawm_cli/, and wildawm_common/. Data and models are on Hugging Face: ChilleD/WildAWM (341 environment-task pairs), WildAWM-35B-A3B, WildAWM-4B.

Positioning in one line: this is not another GUI benchmark. It is a complete pipeline that builds worlds out of open-weight models and then runs online reinforcement learning inside those worlds, and it reduces the question "can we trust a synthesized task?" into five executable gates.

Background and Motivation

The paper starts from a claim that is quoted often and honored rarely: agent learning is shifting from learning off humans to learning off the world. Learning off humans is bounded twice over - demonstrations are expensive, and the ceiling is the demonstrator. Learning off the world requires an executable environment to trial-and-error in, which turns environment supply into the actual bottleneck. Rather than stopping at the slogan, the authors name three concrete failure modes of doing online RL on real computer systems: states are hard to reset (restoring a live machine to an exact historical state is close to impossible), online services drift over time (this week's website is not the environment you trained on last week), and exploratory actions have irrevocable consequences - a footnote even cites OpenAI's incident reporting on real-world RL. Hand-built environments recover controllability and verifiability, but they are expensive and do not scale.

The second motivation is definitional. The authors deliberately adopt a broader notion of computer use than the mainstream one: a real computer task is defined by what has to get done, not by a single interaction modality. The same goal may require a few clicks in a GUI, a script in a terminal, and one MCP service call, so choosing between interfaces and coordinating across them is itself a core capability. Existing synthesis pipelines each watch one modality: browser use, tool calling, terminal workflows. Even computer-use-agent (CUA) work is mostly GUI-centric. Table 1 of the paper lays the spectrum out plainly - AWM, WebArena-Infinity, Gym-Anything, and CUA-Gym are GUI only, TMax and Synthetic Computers are CLI only, CUA-Universe covers GUI plus CLI, and WildAWM is the one that covers GUI, MCP, and CLI together with a scope of the whole personal computer rather than a single application.

The third motivation is easy to skim past, yet it is the sharpest difference from related work: every model used for synthesis is open source. In Table 1 the other seven pipelines synthesize with GPT-5, Opus 4.6, Opus 4.5/4.6, Gemini-3-Pro, Sonnet 4.6, and GPT-5.6; the WildAWM row says OpenSource (GLM-5.2 as both generator and judge, with GLM-5.2, DeepSeek-V4-Flash, and MiniMax-M3 in the rollout ensemble). The authors argue this on grounds of reproducibility, transparency, and accessibility: if building environments requires buying frontier closed API access, then the ceiling of this route is set by somebody else's pricing and rate limits.

Fourth, task synthesis is hard in three specific ways, which the paper states explicitly: a task must be solvable in the synthesized world state; it must be hard enough to produce a useful learning signal; and it must ship with a reliable verifier able to separate "actually done" from "looks done". The authors are unusually firm on the third point - they refuse the simplifying assumption that a frontier model saying so makes it so. That stance produces the execution-evidence quality gates described below, plus one genuinely clever design: using LLM sycophancy in reverse as a stress test for the judges.

The last motivation is about the granularity of realism. Prior methods often synthesize visually simplified applications and treat each one as an isolated environment, whereas real computer use happens in a connected digital workspace - websites, applications, MCP services, and local files coexisting on one executable PC whose state accumulates over time. WildAWM therefore builds worlds from the angle of "one person living through many days": what you can do today depends on the chat threads, inventory sheets, credential files, and return tickets accumulated over the previous ten. The case study later in the paper makes this concrete.

Figure 1: WildAWM overview (paper Figure 1). Stage 1, World Synthesis, uses a coding agent to construct persona-driven multi-day world states whose environment exposes GUI, MCP, and CLI surfaces at once. Stage 2, XAI Task Synthesis, generates cross-interface tasks through five quality gates that test solvability, difficulty, and verifier reliability. The output is a {task, environment, verifier} triple that trains a generalist CUA directly.

Preliminaries: Worlds, Tasks, and Verifiers

Three pieces of notation hold the whole pipeline together. A world is the accumulated state of a personal computer, containing web applications, services reachable over MCP, and local files that record somebody's daily activity. The accumulated state of persona $c$ after day $d$ is written $W(c,d)$; it is both the initialization source of the environment and its reset point. To give multimodal GUI interaction enough visual material, the authors integrate the 15 mirrored WebHarbor sites (AllRecipes, Amazon, Apple, arXiv, BBC News, Booking, Cambridge Dictionary, Coursera, ESPN, GitHub, Google Search, Google Flights, Google Map, HuggingFace, WolframAlpha). That is why Table 1 labels WildAWM's synthesis source as "From-Scratch + Seed-Grounded": the GUI substrate and image content come from seeds, everything else is synthesized from zero.

A task is $\tau$, and it is required to use all three interfaces - the paper calls these XAI (cross-agent interface) tasks. A verifier $V$ is an executable program that takes the initial world state $W_{0}$, the final world state $W$, and the agent's final answer $a$, and returns a checklist of $N$ binary criteria:

$$v_{i}(W_{0},W,a)\in\{0,1\},\qquad V(W_{0},W,a)=\prod_{i=1}^{N}v_{i}(W_{0},W,a)$$

The product form means success requires every criterion to pass, while the score of an individual criterion doubles as fine-grained feedback and as a value estimate for long-horizon RL - the reward design in Equation 1 uses exactly that. Besides $\tau$ and $V$, the generator also produces a rubric $R$, a hint $h$, and a document that decomposes $\tau$ into ordered subtasks. These artifacts drive the gates during synthesis; during training only the programmatic verifier $V$ is consumed as the reward function.

Method

The pipeline compresses into three sentences: build the world, then build the task, then do online RL inside the world; whether a task enters the pool is decided by execution evidence. The diagram below traces the real data flow and failure loops as described in sections 3.1-3.3 and Appendix A.3-A.6.

flowchart TB
    subgraph SG1["Stage 1 - World synthesis: GLM-5.2 on OpenCode"]
        P1["Persona bible
occupation, at least 3 named contacts
2 to 5 multi-day arcs, one image-decisive"]
        P2["Schedule planning
reads W c d-1 for continuity
coverage pressure from site and app histograms"]
        P3["Skill-augmented implementation
matches 83 reusable event skills
implements schema, backend and MCP server if missing"]
        P4["delta.sh deterministic replay
timestamps frozen as literals
primary keys derived from MAX id plus 1, idempotent inserts"]
        P5["Five-layer world validation
static lint, two replays hash-identical, declaration check
live health check, reviewer agent outside the container"]
    end
    subgraph SG2["Stage 2 - XAI task synthesis"]
        T1["Task generation
explores the booted live environment
proposal must use GUI, MCP and CLI"]
        T2["Diversity gate
compared against accepted and in-flight tasks
least similar proposals advance"]
        T3["Four artifacts
task spec, executable verifier V
rubric R, hint h, subtask decomposition"]
        T4["Heterogeneous ensemble rollouts
6 per task, half hinted
3 open models x 6 frameworks = 18 configs"]
        T5["Three-tier adjudication
test-time scaling of verification:
programmatic V, LLM judge, agent judge"]
    end
    subgraph SG3["Five quality gates"]
        G1["Judge consistency
V equals V_LLM equals V_agent on all 6"]
        G2["Initially unsatisfied
V W0 W0 empty set equals 0"]
        G3["Solvability
at least 1 success, every declared interface used"]
        G4["Difficulty
k over 3 unhinted rollouts: k at most 1 is hard
k equal 2 is medium, k equal 3 is rejected"]
        G5["Counterfactual probe
opposite verdict shown to the agent judge
editing verifier scripts counts as caving"]
    end
    subgraph SG4["Stage 3 - Generalist CUA training"]
        R1["SFT warm start
100 GLM-5.2 trajectories, 3 epochs"]
        R2["GRPO online RL
checklist partial credit weight 0.1 over N
format failure scores minus 1"]
        R3["Adaptive rollouts
multiplier capped at 3, appended in blocks of 8
a group must contain two distinct rewards"]
        R4["Segment-wise training
cut at context compaction, keep intra-segment causality
forward passes drop from T to M"]
    end
    P1 --> P2 --> P3 --> P4 --> P5
    P5 -->|640 world states W c d| T1
    T1 --> T2 --> T3 --> T4 --> T5 --> G1 --> G2 --> G3 --> G4 --> G5
    G5 -->|all gates pass, 341 tasks| POOL["Task pool
100 SFT, 208 RL, 33 in-domain eval"]
    G5 -->|any gate fails, with diagnostic evidence| FIX["At most two revision rounds
revised task must clear all gates again"]
    FIX --> T3
    POOL --> R1 --> R2
    R3 --> R2
    R2 --> R4
    R4 --> OUT["WildAWM-4B and WildAWM-35B-A3B
improved on all 8 out-of-domain benchmarks"]

Figure 2: the full WildAWM chain, drawn from the actual structure of sections 3.1-3.3 and Appendix A.3-A.6. Two loops matter: a failed gate returns to the generator with diagnostic evidence for at most two revision rounds, and rejected tasks are not wasted - their lessons are absorbed into the generator's evolving skill library.

1. World synthesis: from one persona to 640 replayable states

World synthesis does not begin with code, it begins with inventing a person. A persona-creation agent receives a constitution (a system prompt fixing deliverables and quality bars) and must produce a persona bible for one concrete, plausible human. The crux is grounding: the agent first inspects the 15 WebHarbor seed databases and the current shared MCP application index, so that the person's digital activity references entities that actually exist in the catalog - real products, real courses, real hotels. On a cold start there are no MCP services at all, so each persona ships a grounding roadmap for four to eight services bound to specific story arcs; missing services are implemented by a dedicated application-creator stage on first use and published to a shared library for later days and later personas. The deliverables also include a structured profile (demographics, at least three named contacts, tools and services, multi-day arcs with at least one image-decisive arc), web account credentials with deterministic password hashes, and a compact signature consumed by the diversity gate.

Diversity is enforced by an LLM judge: a candidate persona is compared against every accepted persona, and it is rejected if it shares the same occupation and life domain, the same core tool or site-usage pattern, or the same narrative arc. Rejected candidates receive diagnostic feedback and may be revised at most twice. Schedule planning is sequential in the same way: the day planner gets the persona bible, all previous schedules, and read access to the accumulated world state through day $N{-}1$, and emits a markdown plan covering computer-use activity, MCP service events with continuity (reply threads, calendar updates), and web GUI actions that reference real catalog entities. The plan also carries a coverage-pressure block - histograms of per-app and per-site usage from previous days - which pushes the planner to spread interface use across days.

Materialization is done by a "god-mode" coding agent running inside a booted live world instance with full access to MCP services, WebHarbor databases, and the filesystem. Its deliverable is a delta.sh (with deterministic Python sub-scripts) that replays closed over the day $N{-}1$ state and reproduces day $N$ identically every time. Determinism is not a convention, it is a set of hard constraints: timestamps must be frozen literals (datetime.now() is never allowed), primary keys must be derived (MAX(id)+1), inserts must be idempotent, and passwords must be hashed with a fixed salt from the persona's accounts.json. When the plan requires a capability that does not exist yet, the implementer may emit an overlay containing a new MCP server or a WebHarbor code patch. All world instances share one application layer but hold isolated copies of mutable state (service databases, site histories, local files).

The scale of the output deserves its own look, because it explains where "median 52.4K tokens per environment" comes from:

Dimension	Value	Dimension	Value
Personas	36	Mirrored sites / MCP services	15 / 8
World states	640	Reusable event skills	83
States per persona	17.8	Sites / MCP services per persona	5.25 / 5.44
Sites / MCP touched per day	3.97 / 5.31	Total database records	171.6K
Tables / files touched per day	15.38 / 4.97	Local files (mean on last day)	50.1
World growth per day	30.9 KiB	Size of one world state	59.3 MiB

Table 1: world synthesis statistics (paper Table 2). "Touched" means the service or database was modified during that day's synthesis. Each persona materializes into 8 to 30 world states, 640 in total; 246 of them host at least one accepted task, with at most three task slots per state.

2. Determinism is engineered: delta.sh and the five-layer world validation

The most common way synthetic worlds fail is not "cannot be generated" but "generates something different every time". Five layers pin the day down. (1) Deterministic static check - the delta script is linted, including a cross-persona identity check that refuses names already reserved by another persona. (2) Replay determinism - delta.sh is replayed twice from the same day $N{-}1$ state and the content hashes of both worlds must match exactly. (3) Declaration check - every database write, web write, account, file, and cross-reference declared in produces.json is verified in the materialized world; declared but missing or unchanged fails the day. (4) Live health - the world is actually booted, every declared web account must log in with that persona's credentials and render its history pages, all active MCP services must load, and the persona's web credentials must be discoverable inside the workspace (otherwise a future solver cannot log in as this person at all). (5) Reviewer agent - an independent coding agent, running on the host outside the implementer's container, checks the materialized snapshot against the schedule plan for MCP events, web GUI state, files, and determinism.

The credential-discoverability clause in layer 4 says a lot about the design stance: it is not asking whether the world is self-consistent, it is asking whether the world is playable by a future solver. Layer 5 puts the reviewer outside the container for the same reason the counterfactual probe exists - the judge must not share context with the judged.

The skill library is a second, quieter quality mechanism. Each event skill specifies the interfaces involved, the state changes to produce, and cross-record and cross-file consistency constraints; the paper's example is "a student enrolls in a Coursera course and completes an assignment", which encodes the workflow plus the required database and file updates. When a schedule introduces activity not covered by an existing skill, the generator synthesizes a new one. Skills and service implementations both go into a shared library reused by later days and personas. Those 83 skills and 8 MCP services are why 36 personas can share 640 states without copying each other.

3. XAI task generation and the diversity gate

Given an initial world state $W_{0}=W(c,d)$, the skill-augmented generator explores the booted environment in situ and proposes XAI tasks grounded in the available application data, service records, and local files, with every task requiring all three interfaces. The diversity gate then selects: only the proposals that are least similar to already-accepted tasks and to candidates being developed by other concurrent workers advance. Counting in-flight work matters - synthesis runs in parallel across processes, so deduplication has to happen before anything is written to disk.

Selected proposals are expanded into the full artifact set: task spec $\tau$, executable verifier $V$, rubric $R$, hint $h$, and the subtask decomposition. The intent is direct - $V$ provides objective adjudication, $R$ gives the LLM judge semantic grounds, $h$ creates the hinted and unhinted rollout pairs of differing difficulty, and the decomposition is what lets the pipeline check whether the declared interfaces were genuinely used.

The shape of the synthesis funnel is worth recording. Figure 2 of the paper splits advancing candidates into Direct (passed on first attempt) and After feedback (passed only after at least one feedback-driven revision at some stage) and stresses that the funnel is monotone. Read together with the revision statistics in Table 4, the two hardest gates are solvability (mean 1.16 revisions) and judge consistency (mean 1.09), while the initial-unsatisfied gate almost never fires (0.05) - the low-grade error of "the task is already complete at the initial state" is mostly eliminated upstream by the determinism constraints of world synthesis.

4. Eighteen heterogeneous ensemble rollouts and three-tier adjudication

Once a task and its verifier exist, heterogeneous agents actually attempt it: successful rollouts are evidence of solvability, success rates characterize empirical difficulty, and the trajectories plus outcomes are used to validate the verifier itself. Each task collects 6 rollouts, half of them hinted, drawn from 18 model-framework combinations - 3 open-source models (GLM-5.2, DeepSeek-V4-Flash, MiniMax-M3) times 6 frameworks, split into three coding agents (OpenCode, Codex, Claude Code) and three claw agents (Nanobot, OpenClaw, Hermes Agent). Using a Cartesian product of models and frameworks rather than one strong model keeps the evidence from depending on any single model's behavioral habits.

Every rollout passes three tiers of adjudication, which the authors call test-time scaling of verification: tier one is the freshly synthesized programmatic verifier $V$; tier two is an LLM judge performing a semantic evaluation over the task, rubric, trajectory, and verifier output; tier three is an agent judge on OpenCode that actively inspects the initial and final environment over multiple turns. Agent judges are expensive, so they are used during synthesis only - to cross-check verdicts and find cases where the verifier needs revision. The reward function during RL is always the programmatic verifier. That division of labor is the point: expensive adjudication manufactures a trustworthy reward function, and the cheap reward function scales the training.

5. Five quality gates: turning "is this a good task?" into executable criteria

Execution evidence alone does not make a task valid. Five gates must all pass before a candidate is accepted.

Gate 1 - judge consistency. For rollout $j$, let the final state and answer be $W_{j}$ and $a_{j}$, and let the LLM judge and agent judge verdicts be $V_{\text{LLM}}^{(j)}$ and $V_{\text{agent}}^{(j)}$. The requirement is

$$V(W_{0},W_{j},a_{j})=V_{\text{LLM}}^{(j)}=V_{\text{agent}}^{(j)},\qquad \forall j\in\{1,\ldots,6\}$$

Any disagreement is returned to the generator together with the relevant evidence, to revise the rubric $R$ and the verifier script $V$. In effect this gate uses "three independent adjudications must coincide" as a consistency test on the verifier.

Gate 2 - initially unsatisfied. It requires $V(W_{0},W_{0},\emptyset)=0$, where $\emptyset$ means the agent took no action at all. Treating the initial state as the final state and an empty answer as the answer must fail; otherwise the task was already "complete" at the moment the world was generated, and any agent scores full marks by doing nothing.

Gate 3 - solvability. Consistency cannot prove solvability, since every rollout may have failed. This gate requires at least one success, $\sum_{j=1}^{6}V(W_{0},W_{j},a_{j})\geq 1$, and additionally every successful trajectory must have used each interface declared at task-generation time - evidence that the cross-interface workflow was actually executed, not that some single-interface shortcut happened to pass.

Gate 4 - difficulty. Empirical difficulty is estimated from the three unhinted rollouts; let $\mathcal{U}$ be their index set:

$$k=\sum_{j\in\mathcal{U}}V(W_{0},W_{j},a_{j})$$

$k\leq 1$ is labeled hard, $k=2$ is medium, and $k=3$ is treated as too easy and sent back to the generator to be made harder. In the open-source code this gate is implemented as an absolute success-count interval: hard is $[0,1]$, medium is $[2,2]$, and at startup the pipeline asserts that the two intervals are contiguous and non-overlapping (bare_hard_max_completed + 1 == bare_medium_min_completed) and that no upper bound exceeds live_test_n=3. A misconfiguration raises instead of silently passing.

Gate 5 - counterfactual probe. The first four gates lean partly on model judgments, including the agent judge's feedback used to revise verifiers. Since LLMs are sycophantic, the authors use that against them: given the verdict $V(W_{0},W,a)$ for a rollout, the agent judge is shown the opposite assertion $1-V(W_{0},W,a)$ and asked to re-investigate the execution evidence, producing $V_{\text{probe}}\in\{0,1\}$. The candidate passes only if $V_{\text{probe}}=V(W_{0},W,a)$. Put differently, a judge that flips its verdict when told "are you sure? it looks like a failure to me" is not trusted.

Candidates that fail any gate go back to the generator with diagnostic evidence for at most two revision rounds, after which they must clear every gate again. Accepted tasks enter the diversity pool; rejected ones feed their lessons into the generator's evolving skill library. The final tally: of 341 tasks, 273 are hard and 68 medium, with a mean of 4.2 subtasks and 3.1 interface switches, and 74.5% of tasks require revisiting an interface used earlier - the agent must carry state across surfaces instead of finishing inside one. Mean success rate of synthesis-time rollouts is only 21.3% (45.6% when hinted), and GLM-5.2 sometimes needs more than 100 steps to finish. Consistent with the solvability and difficulty gates, every accepted task was solved by at least one agent, and no task was solved by all of them.

6. What an accepted long-horizon task looks like

The case in Table 3 of the paper is worth reading line by line, because it drops every abstract constraint above into one concrete scenario. The requester is a freelance audio-video technician (Appendix C persona: Lars Petersen, 41, Minneapolis, who installs video walls, sound systems, and projectors for a sports bar, a church, and a community theater), and he needs a production-grade surge protector for the rack that will hold a video wall. The requester's approval sits in a chat thread from an earlier day in the world's history; the workspace holds a rack inventory sheet, a bill of materials, and product spec sheets accumulated since day 0. The solution therefore depends on state that the world accumulated over time, not on a single snapshot.

The interface flow is MCP to CLI to GUI to CLI to MCP - five subtasks, returning twice to CLI and MCP. First the agent confirms the approval message "Go, order it today" in the direct-message MCP service. Then it writes and runs a terminal script that joins three heterogeneous local sources (CSV inventory, markdown bill of materials, PDF spec sheet), ranks candidate surge protectors by spec match, and emits a shortlist CSV. Next comes the decisive step in the storefront GUI: two candidates are identical on every listed specification and differ only in visible appearance, so the agent must look at the product images to pick the one with a white chassis and a numeric display panel. It then writes a restock note and finally sends a direct message through MCP telling the installation assistant which unit was locked in. Each world-changing subtask is checked by a deterministic script (check_audit.py, check_cart.py, check_note.py, check_dm.py), and success requires all of them to pass.

Figure 3: the decisive GUI observation of the case task (obs_product.png, Table 20 in Appendix C). The two candidate surge protectors tie on every textual specification - outlet count, joule rating, cord length, trip indicator. The only discriminating evidence is in the image: a white chassis and a numeric display panel. This is exactly where the persona requirement of "at least one image-decisive arc" pays off.

Figure 4: the page observation after adding to cart in the same trajectory (obs_cart.png). The verifier never reads what the agent claims; it goes back into the world state and checks whether the cart, the CSV, the note file, and the message log all changed.

The evidence this task presents at the five gates: the three tiers agree on all 6 rollouts ($V=V_{\text{LLM}}=V_{\text{agent}}$); all four verifier scripts fail at the initial state, so $V(W_{0},W_{0},\emptyset)=0$; 1 of the 3 hinted rollouts completes, satisfying solvability; among the 3 unhinted rollouts $k=0$, so it is labeled hard; and under the counterfactual probe the agent judge holds its verdict without modifying any verifier file. That last clause is not rhetoric - in the open-source implementation the probe first takes a writable copy of the verifier script directory and computes a SHA-256 digest; if the digest changes after re-investigation, the outcome is recorded as a cave even when the model claims it defended its verdict.

7. Training: checklist reward, adaptive rollouts, and segment-wise training across compaction

Training uses GRPO with reward from the gate-validated verifier, in a form the authors describe as dense yet outcome-based. For rollout $j$ of task $\tau$, let $e_{j}\in\{0,1\}$ indicate a policy format failure (for example an unparseable tool call). If $e_{j}=1$ the reward is simply $r_{j}=-1$; otherwise:

$$r_{j}=\min\left\{1,\;V(W_{0},W_{j},a_{j})+\frac{0.1}{N}\sum_{i=1}^{N}v_{i}(W_{0},W_{j},a_{j})\right\}$$

Completed tasks earn the full 1, while unfinished trajectories get partial credit from the checklist criteria (weight 0.1), giving long-horizon learning a denser signal. The elegance is that it reuses structure that already existed at gate time: the verifier was already a product of $N$ binary criteria, so turning the product into a weighted sum costs almost nothing and converts an all-or-nothing sparse reward into a reward with progress in it.

Small models frequently produce zero-variance reward groups, from which GRPO gets no learning signal at all. Inspired by DAPO, the authors keep an exponential moving estimate of the required rollout multiplier $\lceil K/G\rceil$ per task ($K$ is the rollout pool size, $G$ the target group size), initialized to 1, capped at 3, and updated with weight 0.8 after each optimization step. When $K>G$, they select from the $\binom{K}{G}$ possible groups one that contains samples from both the lower and the upper half of the reward range, encouraging reward diversity. The advantage is therefore piecewise:

$$A_{j}=\begin{cases}(r_{j}-\mu_{\tau})/\sigma_{\tau},&\sigma_{\tau}>0,\\[2pt] -K^{-1/2},&\sigma_{\tau}=0\ \text{and all}\ e_{j}=1,\\[2pt] 0,&\text{otherwise},\end{cases}$$

where $\mu_{\tau}$ and $\sigma_{\tau}$ are the mean and population standard deviation of rewards over the completed rollout pool of size $K$. The middle branch exists specifically for the case where every rollout scored the same $-1$ because of format errors - extremely common early in long-horizon rollout training. Zeroing it would throw the compute away; assigning a $K$-dependent negative advantage preserves the format-related learning signal. A format error terminates the rollout immediately, and the gradient is applied only to the tokens generated in the offending step.

The adaptive rollout mechanism works as follows. When a task is sampled it first launches $G$ base rollouts, then appends blocks of $C=8$ up to a total of

$$n_{\tau}=\min\left\{\kappa_{\max}G,\;G+C\left(\lfloor\kappa_{\tau}+1/2\rfloor-1\right)\right\}$$

After each block the whole pool is inspected: if all $G$ base rollouts succeeded the task is mastered; if the pool contains at least two distinct rewards, or is entirely format failures, it is ready; otherwise blocks keep arriving until $\kappa_{\max}G$ rollouts are launched, and if the condition still fails the task is exhausted. Mastered and exhausted groups have identical rewards throughout and are dropped from the step; outstanding rollouts of mastered or ready tasks are cancelled. The multiplier is updated at the end of the step:

$$\kappa_{\tau}\leftarrow\begin{cases}1,&\tau\ \text{mastered},\\[2pt] (1-\lambda)\,\kappa_{\tau}+\lambda\left\lceil K_{\tau}/G\right\rceil,&\tau\ \text{ready},\\[2pt] \kappa_{\max},&\tau\ \text{exhausted},\end{cases}\qquad \lambda=0.8$$

The paper walks through a concrete trace. With $G=4$, the first sampling launches 4 rollouts; if all four rewards are 0, a block of 8 is appended; if one of those 8 solves the task ($r_{j}=1$) and another passes half the checklist ($r_{j}=0.05$), the task becomes ready with $K_{\tau}=12$, two rollouts are drawn from the lower half and two from the upper half of the sorted 12 rewards to form $\mathcal{G}_{\tau}$, and $\kappa_{\tau}$ updates to $0.2\cdot 1+0.8\cdot 3=2.6$, so the next sampling starts at 12 rollouts outright. Conversely, if next time those first 4 base rollouts already show distinct rewards, the remaining 8 are cancelled and $\kappa_{\tau}$ drops to $0.2\cdot 2.6+0.8\cdot 1=1.32$, so the following sampling launches only 4. This is a scheduler that steers compute toward tasks that still carry information, not merely dynamic sampling.

For the policy loss, let $o_{t,k}$ be the $k$-th token generated by rollout $j$ at step $t$, with policy distribution $\pi_{\theta}^{t,k}=\pi_{\theta}(\cdot\mid p_{m},c_{m}^{{<}t},o_{t,{<}k})$. The importance ratio is

$$\rho_{j,t,k}(\theta)=\frac{\pi_{\theta}^{t,k}(o_{t,k})}{\pi_{\theta_{\text{old}}}^{t,k}(o_{t,k})}$$

and the objective is GRPO with triple normalization - tokens averaged within a rollout, rollouts averaged within a group, groups averaged within a step:

$$\mathcal{L}(\theta)=\frac{1}{|\mathcal{B}|}\sum_{\tau\in\mathcal{B}}\frac{1}{|\mathcal{G}_{\tau}|}\sum_{j\in\mathcal{G}_{\tau}}\frac{1}{L_{j}}\sum_{m=1}^{M_{j}}\sum_{t\in S_{m}}\sum_{k=1}^{|o_{t}|}\ell_{j,t,k}(\theta)$$

$$\ell_{j,t,k}(\theta)=-\min\big(\rho_{j,t,k}A_{j},\ \mathrm{clip}(\rho_{j,t,k},1-\varepsilon_{\text{low}},1+\varepsilon_{\text{high}})\,A_{j}\big)-\beta\,\mathcal{H}\big(\pi_{\theta}^{t,k}\big)$$

Here $\mathcal{B}$ is the set of ready tasks in the step, $L_{j}$ is the trainable token count of rollout $j$ (format-failed rollouts count only the offending step), the clipping is asymmetric with $\varepsilon_{\text{low}}=0.2$ and $\varepsilon_{\text{high}}=0.28$, the entropy coefficient is $\beta=0.001$, and there is no KL penalty.

The last training detail addresses a structural problem unique to long-horizon rollouts. When context approaches the window limit, the OpenCode framework automatically compacts: earlier turns are summarized into a compressed representation and generation continues with a new prefix. That makes arbitrarily long rollouts possible under a fixed context budget, but the trajectory is no longer a single causal sequence. The authors define a segment as a maximal run of consecutive steps whose context forms one causal chain with no compaction inside it; compaction events cut rollout $j$ into $M_{j}$ segments, each sharing a base prefix $p_{m}$. The conventional per-step decomposition treats every step as an independent sample carrying the full framework context $x_{t}$:

$$\mathcal{L}_{j}^{\text{step}}(\theta)=-\frac{A_{j}}{L_{j}}\sum_{t=1}^{T_{j}}\log\pi_{\theta}(o_{t}\mid x_{t})$$

It costs $T_{j}$ forward passes, re-encodes heavily overlapping prefixes, and discards the autoregressive dependency between adjacent steps that share one context window. The segment-wise formulation groups steps by shared prefix and keeps intra-segment causality:

$$\mathcal{L}_{j}^{\text{seg}}(\theta)=-\frac{A_{j}}{L_{j}}\sum_{m=1}^{M_{j}}\sum_{t\in S_{m}}\log\pi_{\theta}\!\left(o_{t}\,\middle|\,p_{m},\,c_{m}^{{<}t}\right)$$

Three benefits follow: forward passes drop from $T_{j}$ to $M_{j}$ (in practice $M_{j}\ll T_{j}$), avoiding re-encoding prefixes; steps inside a segment condition on the model's own earlier outputs, matching autoregressive generation at inference; and normalizing by $L_{j}$ at trajectory level gives rollouts with frequent compaction (many short segments) and rare compaction (few long segments) the same total gradient weight. The paper's example is a 9-step rollout with one compaction at step 5: per-step decomposition yields 9 independent samples, segment-wise yields 3 segment samples.

The implementation settings are also fully disclosed. Synthesis and training both run on 8 NVIDIA B200 GPUs; during synthesis GLM-5.2 acts as both generator and agent judge on the OpenCode framework; evaluation uses OpenCode with GUI tools (click, type) and MCP tools (Playwright) at high reasoning effort; maximum steps are 256 for most benchmarks and 512 for OSWorld 2.0 and ALE; context is 102,400 tokens with automatic compaction; each rollout is capped at 30 minutes, rollouts lost to sandbox or framework errors are retried up to 3 times in a fresh sandbox, and timed-out rollouts still have the verifier run against their final world state so they earn a valid reward. Training first collects 100 GLM-5.2 trajectories on the SFT split to warm-start the Qwen base for 3 epochs, then runs up to 64 RL steps with a batch of 8 tasks per step and $G=4$ rollouts per task. The MoE router and the vision encoder are frozen and everything else trains; environments and training are built on Orchard and slime respectively. The 341 tasks are split at random into 100 SFT, 208 RL, and 33 in-domain evaluation.

8. How the paper's method maps onto the open-source code

The code here is not a reference implementation attached to a paper, it is a runnable version of the pipeline, and several key mechanisms can be matched line by line. First, the counterfactual probe is two-stage and anti-cheat. synthesis/wildawm/synthesis/tasks/probe.py runs a blind re-judgment first (a fresh session with no generator context, calling the heavy judge model through a proxy), and only when blind agrees with the ensemble verdict does it proceed to the gaslight stage, which presents the opposite conclusion using OPP = {"completed": "NOT completed (failed)", "failed": "actually completed"}. If blind already disagrees, it returns blind_mismatch directly; the code comment is blunt about why - that signal is for the fixer's verifier and rubric, "so don't gaslight".

# probe.py: take a writable copy of the verifier scripts and hash them before probing
def _dir_digest(scripts_dir: Path) -> dict[str, str]:
    out = {}
    for p in sorted(scripts_dir.glob("*.py")):
        if p.name == "_verify_sdk.py":
            continue
        out[p.name] = hashlib.sha256(p.read_bytes()).hexdigest()
    return out

# compare digests after re-investigation: edited files mean a cave,
# even if the model claims it defended its verdict
after0 = _dir_digest(work / "verifier" / "scripts")
caved_files = sorted(set(before0) ^ set(after0)) + \
    [k for k in before0 if k in after0 and before0[k] != after0[k]]
# Changed files mean a cave, even if the model claims to defend.
if caved_files:
    outcome = "cave"

Second, the difficulty gate is an absolute success-count interval with self-checks. orchestrator.py::_configured_bare_completion_range reads LIVE_TEST_N=3, BARE_HARD_MAX_COMPLETED=1, BARE_MEDIUM_MIN_COMPLETED=2, and BARE_MEDIUM_MAX_COMPLETED=2 from const_config.py, taking $[0,1]$ for hard and $[2,2]$ for medium, and enforces that the intervals are contiguous, non-overlapping, and bounded by the actual number of bare rollouts - otherwise it raises RuntimeError. The paper's sentence "$k=3$ is too easy and goes back to the generator" is, in implementation terms, "$k=3$ falls outside every legal interval". run_ensemble_and_gate() is the function that wires the 18-config ensemble, the three adjudication tiers, and the five gates together.

Third, the verifier runs each script in a sandbox and aggregates the last RESULT_JSON= line. verifier.py::run_verifier takes verifier/scripts/*.py (excluding the shared SDK _verify_sdk.py), executes each one in the sandbox, and aggregates the overall verdict, per-script results, and script errors; _parse_result_json scans stdout backwards for the last line starting with RESULT_JSON=. _verify_sdk.py provides standard-library-only comparison primitives (for instance difflib.ndiff for before-and-after text), which means the verifier depends on no model at all - it can be called cheaply at every RL step precisely because it is just Python. On the host side the entry points are a set of CLI commands that can be copied verbatim for reproduction:

# World synthesis: create persona -> plan and materialize day 0..14 -> prebake
uv run wildawm syn profile --name "Riley Nguyen"
uv run wildawm syn world --profile <profile-id> --to_day 14
uv run wildawm syn dayplan --profile <profile-id> --day 0
uv run wildawm syn dayimpl --profile <profile-id> --day 0
uv run wildawm syn prebake --profile <profile-id> --to_day 0 --verify_rebake yes

# Task synthesis: one task from a single world state / batch by difficulty
uv run wildawm syn task --profile <profile-id> --day 5
uv run wildawm syn task --profile <profile-id> --days 3,5,7 \
    --difficulty hard,hard,medium --count 3 --parallel 3

# Validation and evaluation
uv run wildawm verify --dir <run-dir> --mode heavy
uv run wildawm eval prepare --benchmark wildawm
uv run wildawm eval run --benchmark cua_gym --limit 2

Experimental Results

Evaluation spans 8 out-of-domain benchmarks across 3 agent classes: web agents on Online-Mind2Web (O2W) and WebArena (WA); claw agents on QwenClawBench (QCB) and ClawEval (CLE); computer-use agents on CUA-Gym (CUG), OSWorld-Verified (OSV), OSWorld 2.0 (OS-2), and Agents' Last Exam (ALE). O2W, WA, and CLE report pass rates, the rest report average scores. Every model is evaluated under the same OpenCode framework, plus self-reported numbers from two prior works for reference (weights unavailable or evaluated with legacy GUI frameworks, so reference only).

Model	Framework	O2W	WA	QCB	CLE	CUG	OSV	OS-2	ALE	Avg
Frontier models
GLM-5.2	OpenCode	86.67	67.26	64.44	59.67	78.78	62.46	45.00	41.39	63.21
Opus-4.8	OpenCode	91.01	71.24	64.74	76.33	84.52	68.58	48.32	46.11	68.86
Opus-5	OpenCode	96.60	75.66	55.02	77.67	84.39	74.64	52.79	49.85	70.83
GPT-5.5	OpenCode	72.22	70.80	62.87	63.67	82.71	61.46	47.12	52.39	64.16
GPT-5.6-Sol	OpenCode	86.72	67.70	61.34	64.33	82.83	58.48	46.89	53.45	65.22
GPT-6-Astra	OpenCode	92.78	74.34	65.21	80.33	83.56	71.28	60.77	56.58	73.11
4B class
Qwen3.5-4B (base)	OpenCode	26.39	30.09	44.06	26.67	21.39	18.18	0.87	7.00	21.83
WildAWM-4B	OpenCode	45.65	39.82	48.37	36.67	37.35	22.48	1.03	8.96	30.04 (+8.21)
35B-A3B class
Qwen3.6-35B-A3B (base)	OpenCode	69.60	53.54	57.86	38.00	53.73	33.16	13.22	23.57	42.84
OpenForgeRL (self-reported)	OpenClaw	-	-	33.7	45.1	-	-	-	-	-
CUA-Gym (self-reported)	R-GUI	-	44.5	-	-	-	62.1	-	-	-
WildAWM-35B-A3B	OpenCode	80.77	67.26	61.44	54.67	65.95	49.22	21.73	26.31	53.42 (+10.58)

Table 2: main results on the 8 out-of-domain benchmarks (paper Table 5). O2W, WA, and CLE are pass rates, the others average scores; the two self-reported rows are numbers from the original papers, included for reference only.

The first conclusion is the breadth of transfer. Training only inside these synthetic worlds improves both model sizes on all 8 benchmarks, by +10.58 and +8.21 on average. The operative word is "all": the training environments are synthetic personal-computer worlds while the evaluation is real WebArena, real Online-Mind2Web, real OSWorld. No evaluation content leaked into the synthetic worlds, so the gains can only come from capability.

The second is that the larger model benefits more. The 35B-A3B average gain exceeds the 4B one, and the gap is pronounced on four benchmarks: CLE +16.67, OSV +16.06, WebArena +13.72, CUG +12.22. The authors read this as evidence that WildAWM's long-horizon, cross-interface supervision does not attenuate with capacity and may in fact be exploited more fully by it. The engineering implication is direct: if the value of synthetic data grows with model scale, then "build the environments first, wait for the models to get bigger" is a rational order of investment.

The third is a sharply narrowed gap to frontier models. The 35B-A3B average goes from 42.84 to 53.42; after training the model matches GLM-5.2 on WebArena (67.26 each) and beats GPT-5.5 on Online-Mind2Web (80.77 vs 72.22). Note the measurement discipline: these comparisons are made under the same OpenCode framework and the same high reasoning effort, not against numbers reported in other papers under other harnesses.

Synthesis quality: bigger and more diverse environments, harder tasks, trustworthy verifiers

Category	Metric	Mean	Median	P90
Task corpus	Total tasks (hard / medium)	341 (273 / 68)
	Tasks per persona / per world	9.5 / 0.53
	Interface switches per task	3.1
Synthesis-time success rate	Unhinted SR	21.3%
	Hinted SR	45.6%
Revisions per gate	Initially unsatisfied	0.05	0	0
	Judge consistency	1.09	1	3
	Solvability	1.16	1	3
	Difficulty	0.39	0	1
	Counterfactual probe	0.18	0	1
Synthesis cost per task	Revision rounds	2.5	2	5
	Generator input tokens (millions)	69.8	58.3	130.9
Task execution (GLM-5.2 + OpenCode)	LLM calls / tool calls	80 / 81	74 / 76	121 / 124
	Input tokens (millions)	4.1	3.7	6.5
	Sites visited / MCP connected / files handled	1.1 / 2.4 / 5.0	1.0 / 2.0 / 4.0	2.0 / 3.0 / 9.0

Table 3: XAI task statistics (paper Table 4). SR is the mean success rate over synthesis-time rollouts across the 18 agent configurations; "revisions" counts feedback-driven revision rounds at each gate.

On the environment side: a WildAWM world state is the accumulation of a persona's multi-day activity with a median size of 52.4K tokens, whereas CUA-Gym uses task-specific injected states with a median of 1.1K tokens - nearly a 50x difference. The extra scale does not cost diversity: representing environment artifacts with Qwen3-Embedding-8B, WildAWM has a consistently higher Vendi score at every sampled size (non-overlapping 95% confidence intervals) and a larger nearest-neighbor cosine distance, 0.25 vs 0.20, indicating less redundancy. The authors stress that these numbers were obtained using open-source models only.

On the task and verifier side, the paper answers four questions about itself. (1) Are the tasks solvable? Not by endorsement from one strong model - 24 heterogeneous agents (the 18 configurations plus Opus-4.8 and others) are used, every task is solved by at least one agent, and the three open-source synthesis models alone solve more than 95%; with hints every task is solved by at least one of the three, and 51.5% are solved by all three. (2) Are they hard enough? GPT-6-Astra reaches only about 60% pass@1, and difficulty survives test-time scaling - even with eight attempts it still fails 27.3% of tasks, and pass^k falls quickly, so these tasks still carry training signal for frontier models. (3) Is the verifier reliable? On 100 random rollouts compared against several agent-as-a-judge models, agreement is 95% with Cohen's $\kappa$ of 0.90, identical to the majority-vote result of the judge panel (0.95 / 0.98 / 0.93 / 0.95 / 0.90), while agreement among judges is uneven - $\kappa$ is 0.94 between GLM-5.2 and Opus-5 but only 0.79 between GPT-5.6-Sol and Opus-5. (4) Does it scale? Synthesis continued to 134 domains and 91 task types at a 53% acceptance rate with no visible saturation, and the mean pairwise cosine distance stays stable as the pool grows - coverage was not bought by emitting ever more similar tasks.

Gate ablation: the five gates are complementary, not redundant

Setting (Qwen3.5-4B)	O2W	QCB	OSV	In-domain Val.	Avg
Base	26.39	44.06	18.18	0.38	22.25
WildAWM + SFT (all gates)	38.49	50.17	17.17	21.59	31.86
Without the initially-unsatisfied gate	33.21	42.36	17.71	3.05	24.08
Without the judge-consistency gate	29.10	37.72	19.62	3.79	22.56
Without the solvability gate	44.87	44.15	12.12	15.91	29.26
Without the difficulty gate	39.05	42.29	16.54	7.58	26.37
Without the counterfactual probe	31.37	42.55	24.09	13.64	27.91

Table 4: quality-gate ablation (paper Table 7), success rates in %, Val. is the synthesized in-domain validation set. One gate is removed at a time; every other setting is identical.

The protocol removes one gate at a time, re-synthesizes the SFT data under the same settings, and trains Qwen3.5-4B. The result is unambiguous: the full gate set is best, and removing any single gate costs 2.60 to 9.30 points on average. The two most load-bearing are judge consistency and initially-unsatisfied - without them the average drops to 22.56 and 24.08, and the in-domain validation set collapses from 21.59 to 3.79 and 3.05, essentially back to base level (0.38). Read together, those two numbers are persuasive: the in-domain set measures whether the model learned these tasks at all, so a collapse means the data contained many samples whose reward signal was simply wrong - tasks already complete at the initial state, or verifiers disagreeing with the judges - and the model was trained to fit noise. Removing the difficulty gate produces the same pathology, with in-domain down to 7.58.

The two gates whose removal hurts less are equally informative. Without the solvability gate, O2W actually rises to 44.87 (above the full pipeline's 38.49) and the average falls only 2.60; without the counterfactual probe, OSV rises to 24.09 (full pipeline 17.17) and the average falls 3.95. In other words these two gates are noisy on any single out-of-domain benchmark; their value shows up in in-domain consistency and in the overall average. That matches their function: both exist to prevent untrustworthy verifiers, and the cost of an untrustworthy verifier is systemic rather than visible on one benchmark.

RL ablation and behavior analysis: SFT teaches interfaces, RL teaches horizons

Setting (Qwen3.6-35B-A3B)	O2W	WA	QCB	CLE	CUG	OSV	OS-2	ALE	Avg
WildAWM (full recipe)	80.77	67.26	61.44	54.67	65.95	49.22	21.73	26.31	53.42
Without the format penalty	-	-	-	-	-	-	-	-	51.17 (-1.24)
Without checklist partial credit	-	-	-	69.89 pass rate	-	-	-	-	51.17 (-2.25)
Without the SFT warm start	66.29	57.96	59.74	51.67	68.34	42.78	13.54	24.59	48.11 (-5.31)

Table 5: RL recipe ablation (paper Table 8), showing the average column plus the per-benchmark comparisons named in the paper text.

Three observations. Dropping the format penalty clearly hurts WA and OSV; CLE pass rate is actually higher without it (69.89 vs 69.45) yet the average is not, which means more tasks merely scrape over the pass line. The authors infer that format reward matters especially for GUI agents, whose actions must parse correctly before they can touch a visual interface. Removing checklist partial credit lowers most benchmarks, confirming that a denser reward improves credit assignment. And removing the SFT warm start causes the largest regression (-5.31), which at this model scale means a good initialization is close to decisive for whether RL gets off the ground at all.

The behavior analysis (paper Figure 5 and Appendix B) separates cleanly into what SFT teaches and what RL teaches. The comparison involves four variants: SFT (fine-tuned on GLM-5.2 trajectories, i.e. the WildAWM initialization checkpoint), RFT (fine-tuned on the base model's own successful trajectories), +RL (the same RL recipe without SFT), and full WildAWM. First, gains concentrate on tasks the teacher can reach. Splitting out-of-domain tasks into four difficulty quartiles by how many frontier models solve them, WildAWM gains most in the second and third quartiles (+19.9 and +12.2), with RL adding 5.0 and 1.3 on top of SFT, while +RL alone gets 6.7 and 8.7. In the hardest quartile (hard for the teacher GLM-5.2 as well) no variant gains meaningfully. Second, cross-interface behavior is learned from the teacher. The interface switch rate rises from 18.4% at base to 24.7% after SFT and 25.5% after RL, approaching GLM-5.2's 26.3%, whereas RFT and +RL push it down to 15.8% and 15.7%. The share of GUI actions among tool calls falls from 16.3% to 4.3% (1.6% after RL) while RFT largely keeps the base habit at 17.2% - what the student learns is the teacher's working style of clicking less and using terminals and services more.

Third, RL's contribution is horizon, not in-domain fitting. On WildAWM-val, SFT already matches the teacher (37.3% pass@1 vs 37.9%) and RL lands at 36.0%, no higher; yet RL lifts the net out-of-domain gain from +93 tasks to +129 (RFT reaches only +73). RL also removes a long-turn pathology that SFT creates and the teacher does not have: the share of turns emitting at least 8K tokens is 24.3% for SFT, 9.1% after RL, and 7.4% for GLM-5.2 (base 12.8%, RFT 11.7%). The step-budget view agrees: within 40 steps SFT and WildAWM solve a similar share (34.9% vs 36.8%), but between 40 and 120 steps SFT gains only 9.7 points while WildAWM gains 17.0, close to the teacher; +RL also gains 15.2 but from a lower start, reaching only 47.2% within 120 steps against WildAWM's 53.7%. Per benchmark, that extra gain comes from CUG (18.0 vs 7.5), OSV (16.2 vs 8.1), and WA (16.4 vs 12.4). The authors also note that counting timeouts as unsolved is conservative for WildAWM: 5.5% of its episodes on these benchmarks time out before 120 steps.

Limitations

1. Stated by the authors: the longest tasks remain out of reach. The hardest quartile in Figure 5 consists mainly of long OS-2 and ALE tasks, on which the base model and its four variants together solve only 4 to 7, against 18 for the strongest reference GPT-6-Astra. The authors explicitly list this as future work: given compute, study whether larger models can exploit this synthetic data better and perform more strongly on such tasks. It is the same fact seen from two sides as "gains concentrate on tasks the teacher can reach" - the ceiling on the value of synthetic data is currently set by the capability ceiling of the teacher (GLM-5.2).

2. Synthesis cost is tabulated but not amortized. Each task consumes on average 69.8M generator input tokens (median 58.3M, P90 130.9M) and 2.5 revision rounds, plus 6 rollouts of up to 30 minutes each averaging 80 LLM calls and 4.1M input tokens; a 53% acceptance rate means nearly half the compute goes to candidates that are ultimately rejected, and the whole pipeline runs on 8 B200 GPUs. The paper reports neither GPU-hours per point of improvement nor an equal-budget comparison against training directly on real environments or human-annotated data, so the cost-effectiveness of this route can only be judged qualitatively today.

3. The corpus distribution is visibly skewed by synthesis order. Of the 341 tasks, the 6 earliest personas contribute 178 (52%), because the pipeline was developed and debugged on them - they had more days available for task generation and more attempts; the other 30 personas contribute 163 in total, between 2 and 14 each. Only 246 of the 640 world states host at least one accepted task. This is not concealment - the authors point out the concentration themselves in the appendix - but it means the effective diversity of "341 tasks" is lower than the headline number, and the training distribution is weighted toward a handful of life domains.

4. Verifier reliability is measured relative to judges, and the judges disagree. Agreement of 95% and $\kappa=0.90$ against the judge majority is a handsome number, but the same table shows $\kappa$ of only 0.79 between GPT-5.6-Sol and Opus-5. The reference frame for "reliable" therefore contains substantial internal disagreement, and 95% is better read as "reliable within the natural variance of strong, heavy judges". In addition, the agent judge in the three-tier adjudication is used only during synthesis, while training rewards come entirely from the programmatic verifier - if a task's verifier has a hole the gates did not catch, RL will exploit it consistently, and the paper reports no reward-hacking rate during training.

5. In-domain capability does not improve under RL, and the generalization mechanism is not explained. On WildAWM-val RL lands at 36.0%, slightly below SFT's 37.3%, while the net out-of-domain gain rises from +93 to +129 tasks. The authors interpret this as RL's benefit lying in transfer rather than in-domain fitting, but the paper offers no mechanistic account of why a model that stops improving on synthetic tasks solves 36 more real-benchmark tasks - whether the driver is better use of the long-horizon step budget, a change in interface-coordination strategy, or something else. That causal link currently rests on behavioral statistics alone.

Summary and Outlook

The real contribution of WildAWM is not "another batch of synthesized environments"; it is that the trustworthiness problem of environment synthesis has been engineered. It assumes neither that the generator is right nor that a frontier model is right, and instead hands adjudication to executable evidence: deterministic replay guarantees the world can be reset with identical content hashes across two replays; the five-layer world validation guarantees state matches declarations and services actually log in; 18 heterogeneous rollout configurations guarantee that solvability and difficulty are not one model's opinion; three-tier adjudication guarantees the verifier coincides with two independent judges; and the counterfactual probe turns sycophancy into an instrument for testing how stiff a judge's spine is - even "the judge quietly edited the verifier script" is caught by a SHA-256 digest. The gate ablation numbers (2.60 to 9.30 points lost on average for any removed gate, in-domain validation collapsing from 21.59 to 3.05) show these gates are not ceremony: they determine whether the reward signal in the training data is signal or noise.

The second layer of value is that "can open-source models build good environments?" gets a direct answer. Every comparable pipeline synthesizes with GPT-5, Opus, Gemini, or Sonnet; WildAWM uses GLM-5.2, DeepSeek-V4-Flash, and MiniMax-M3 throughout and still produces environments with a median of 52.4K tokens that beat CUA-Gym on both Vendi score and nearest-neighbor cosine distance, plus tasks on which GPT-6-Astra reaches only about 60% pass@1. For anyone who cares about reproducibility and cost this matters more than the +10.58 points: it takes world-building out of the pricing power of closed APIs.

The third layer is a set of unglamorous but concrete engineering judgments on the training side: the reward reuses the verifier's checklist structure for partial credit (weight 0.1/N), zero-variance groups keep format signal via $-K^{-1/2}$, adaptive rollouts steer compute toward tasks that still carry information, and segment-wise training cuts at context compaction to preserve intra-segment causality while dropping forward passes from $T_{j}$ to $M_{j}$. All of these are problems specific to long-horizon agent RL and simply do not arise for short tasks - and they are exactly where someone reproducing this route is most likely to trip. Fortunately the code and configuration are public.

Three next steps would be most informative: swap in a stronger teacher and see whether the hardest quartile budges (already listed as future work); report GPU-hours per point of improvement with an equal-budget comparison against training on real environments, so cost-effectiveness becomes judgeable; and report reward-hacking and verifier-miss rates during RL, upgrading "the verifier is reliable" from agreement with judges to robustness against an adversary that is actively optimizing against it. If those three hold, "mass-produce executable worlds with open-source models and run online RL inside them" could become a standard training paradigm for computer-use agents rather than one successful experiment.

Golden Quotes