OPEN SOURCE DEEP DIVE
OpenMausBot: A Local-First Control Plane for Multi-Agent Computer Use
A local-first control plane for multi-agent computer use, with durable delegation, provider-specific approvals, isolated browsers, narrow remote access, and headless operation.
Most desktop agent products ask the model to do more inside one chat window. OpenMausBot takes a different route. It turns the local computer into a shared operating surface for several bots, each backed by a different agent engine, and gives the human a single place to observe their work, answer questions, approve risky actions, and control where each bot is allowed to act.
The repository is an Electron and React application wrapped around a local harness. That harness owns the provider processes, exposes typed commands over HTTP, and folds their different event formats into one server-sent event stream. This separation matters because Claude Code, Codex, Grok Build, custom ACP agents, and other compatible engines do not share a native protocol. OpenMausBot makes the orchestration layer, rather than a particular model vendor, the stable boundary.
A two-process architecture with one event contract
The desktop renderer talks to a harness listening on the loopback address 127.0.0.1:8799. The renderer holds no provider transport of its own. It sends commands such as creating a bot, starting a turn, answering a permission request, or changing a model, and it consumes one normalized event stream. Behind that boundary, a driver registry maps each configured bot to a live provider instance. Claude, Codex, Grok Build, and ACP-based agents are adapted into the same runtime vocabulary.
The normalized stream is more than a log. It carries enough structure for the application to know whether a model is reasoning, calling a tool, waiting for approval, producing a final answer, or has ended the turn. The provider remains responsible for its native reasoning and tool mechanics, while the harness owns lifecycle, fan-in, persistence, and the permission broker. A provider that cannot start does not take down the rest of the fleet. Its driver degrades to an unavailable state instead of crashing unrelated conversations.
| Layer | Owns | Why the boundary is useful |
|---|---|---|
| Desktop application | Chat, model selection, approvals, computer panels, settings | The user can change engines without changing the interaction model. |
| Harness API | Turns, bots, threads, permissions, computer lifecycle, connectors | Every client sees one command surface and one event stream. |
| Provider drivers | Native CLI sessions and protocol translation | Vendor-specific behavior stays isolated behind a small contract. |
| Assignment runtime | Delegation trees, queues, questions, budgets, restart recovery | Multi-bot work has durable state rather than a pile of promises in the UI. |
Delegation is treated as a durable job system
The most consequential part of the repository is its Assignment runtime. When one bot asks another bot to do work, the request becomes a stored record with a root, parent, depth, requester, assignee, approval level, status, result, evidence, and event history. That record survives process restarts. It is not reconstructed by asking a model what it remembers.
The lifecycle includes queued, running, waiting for child work, waiting for human input, ready to resume, completed, failed, and cancelled states. A parent that delegates several tasks goes into a waiting state, then resumes once with the collected results. If a child asks a question, the parent can be resumed with that question rather than inventing an answer. External runtimes can poll assignment status, and a one-way notice can be sent without creating a dependency.
The runtime also has explicit guards against runaway collaboration. The root of an assignment tree carries budgets for requests, executions, and wakes since its last durable progress. The defaults in server/runtime/types.ts allow a depth of four, twenty-four requests, forty-eight executions, and forty-eight wakes. A queued task expires after twenty-four hours, a tree is failed after four hours without durable progress, and one turn is considered stalled after twenty minutes without activity. These are operational defaults, not claims that every team should use the same limits.
| Guardrail | Default | Scope |
|---|---|---|
| Delegation depth | 4 levels | Maximum parent-child depth in one assignment tree |
| Request and execution budgets | 24 requests, 48 executions, 48 wakes | Counted from the last durable progress in the tree |
| Queue expiry | 24 hours | Queued work that has not started |
| Stall detection | 4 hours per tree, 20 minutes per turn | Separate limits for the whole tree and one active turn |
Restart handling is unusually concrete. A turn cut before it dispatched children may be rerun once in place. A turn cut after children started returns to a waiting state and resumes from their outcomes. A turn that the host knows already performed an outward action, such as sending a message or pushing a commit, is not rerun. Each reconciliation writes an event. This is the kind of bookkeeping that agent systems usually leave implicit until a laptop sleeps in the middle of delegated work.
Approval is a provider setting, not a universal safety claim
OpenMausBot passes one of four conceptual permission levels to its engines: ask, auto-accept edits, approve for me, and full access. Each maps to native provider modes where possible. The project is candid that the same label does not always mean the same mechanism. Codex, Claude, Grok Build, Qwen Code, Gemini CLI, Droid, and custom ACP agents implement different sandbox and review capabilities. The harness records the effective level on the assignment and caps inherited authority according to what the receiving engine can actually support.
The default level is full access. That is a serious deployment choice, not a neutral convenience. In this mode the provider receives its most permissive native setting, residual permission prompts may be answered automatically, and actions such as connected-app sends can proceed without another confirmation card. Full access still does not bypass operating-system privacy controls, sign-in challenges, service permissions, or workspace grants. It also does not answer real missing-information questions. The repository states these limits directly, which is important: a permission modal is not the same thing as a security boundary.
A Chief of Staff can pass its approval level down to delegated work, but only upward in the hierarchy: a lower-level teammate does not wind up with less authority because an ordinary peer delegated something. The receiving engine may cap a requested level. The runtime stores the resolved level with the assignment and checks that the turn actually ran under that level. This does not make a risky workflow safe by itself, but it gives an operator a durable record of which authority was exercised.
One bot roster, several kinds of computer
OpenMausBot separates the agent from the place where it acts. A bot can work in a cloud Linux desktop, an isolated local virtual machine, an automation browser, Android tools, connected applications, or a supported host computer. This distinction is central to the design. A shared model account should not automatically mean shared access to every desktop, browser profile, credential, and application.
The browser path runs each bot with its own profile and Chromium instance. All connections pass through a per-session SOCKS5 egress gate. The gate resolves hostnames itself and checks every resolved address, so a redirect or DNS-rebinding attempt cannot quietly turn a public page into a request to the management API. Loopback access to OpenMausBot's own services is blocked. Cloud metadata addresses, link-local ranges, multicast, and special-use networks are denied. On self-hosted servers, an operator can explicitly allow selected private networks or ports. If the gate is unavailable, the browser fails closed rather than bypassing the policy.
This isolation is not a complete security sandbox for model reasoning. It constrains where the bundled automation browser may connect. Commands run by an engine that already has filesystem or shell access are governed by that engine's own sandbox and by the approval level. The repository makes the distinction visible instead of presenting every computer surface as equivalent.
Host control is deliberately uneven across platforms
On macOS, the packaged application can control the host computer after explicit opt-in. On Ubuntu x64, local host control is beta and available on GNOME Xorg after a second explicit enablement. The bundled Cua Driver runtime is pinned, hash-checked during packaging, and started without its decorative full-screen cursor overlay. On GNOME Wayland, host control is disabled. Preview may work through the desktop portal, but the app does not route around the Wayland safety gate through XWayland.
That asymmetry is a strength of the documentation. It would be easy to advertise "computer use on Linux" by treating a portal preview as equivalent to real-seat keyboard and pointer control. The repository refuses that shortcut. The Ubuntu guide describes the gap, the expected Cua runtime, and the cleanup behavior for old opt-ins. Cloud desktops and local virtual machines remain available on both Xorg and Wayland, so the product still has useful execution surfaces while native input control remains incomplete.
The warning matters for operators building unattended workflows. A browser automation task, a cloud desktop task, and a task that clicks the owner's actual desktop are not interchangeable even if all three appear under a generic "computer" heading. The deployment should choose the narrowest surface that can complete the job and require a person to grant the broader one.
Remote access is a narrow companion API
The phone and desktop companion do not expose the full harness. A separate sidecar receives a paired device request, authenticates a per-device token, checks a default-deny route allowlist, and scrubs response fields before forwarding the request. The harness itself remains on loopback. Pairing uses a short-lived code and a per-launch token in the URL fragment. A device token is not enough to obtain control of a computer: the owner must separately grant computer access for that device.
The allowlist is method-and-path specific. A paired device can read chats and approvals needed by the companion UI, while route families for credentials, arbitrary settings, destructive account actions, local virtual machine lifecycle, and host-only browser surfaces stay closed. New harness routes are closed until someone explicitly adds them. This is a sensible default for a product whose remote surface will keep expanding.
The desktop client can also pair to another host. In that mode the same companion boundary applies, so client mode does not become a proxy for the complete local API. The design is less dramatic than turning the desktop app into a remote shell, but it is much easier to reason about when a lost phone or stale pairing must be revoked.
Headless operation turns the team into automation
The openmausbot command can run a team without the desktop window. A command-line user can apply a team definition, hand one job to a bot, wait for that bot and every teammate it asked, and export the result. A JSON output mode and distinct exit codes make the command usable from CI and ordinary scripts.
The project also ships a stdio Model Context Protocol server. External clients can inspect bots and channels, read compact transcript pages, create bots, send work, wait for completion, switch models, and interrupt turns. The MCP surface intentionally omits approval grants, deletion, arbitrary settings, credentials, and computer lifecycle. That line is useful: an external agent may coordinate the team, but it cannot quietly become the owner of every stored secret or device.
Connected applications use Composio sessions. One installation can hold several labeled accounts for the same provider, which avoids silently replacing the first account when a second one is authorized. A bot receives an explicit tool allowlist instead of inheriting every connected service. That still leaves a substantial trust question: a read-and-write Gmail or Slack action can cause real-world effects, and the approval level governs whether the bot asks before using it. Teams should scope tool grants around a role, not merely around the person who happened to connect the account.
What the screenshots and repository support, and what they do not
The repository demonstrates a coherent architecture and a working early product. It does not publish a controlled evaluation showing that multi-agent teams produce better results than one agent with a longer context. It also does not establish that every provider behaves identically under a shared approval label. Those are independent questions. OpenMausBot's contribution is infrastructure for making the differences explicit: which engine ran, which computer it used, what authority it had, which child assignments it created, and what evidence the harness observed.
Operators should treat the current build accordingly. Start with a local workspace, use Ask or provider-specific review for actions that touch important systems, keep secrets outside prompts, and grant remote computer access only to devices that need it. The default full-access posture is convenient for experimentation but is not appropriate for an unattended administrator. A team that delegates real code, email, or desktop changes should define completion criteria and inspect assignment evidence rather than relying on a fluent final message.
Where the project fits
OpenMausBot belongs in the emerging category of agent harnesses and computer-use control planes. It is relevant to embodied AI teams that need a reusable local runtime for tool-using agents, but it does not control a robot arm or publish a locomotion policy. Its practical examples are software work, browser research, desktop operations, connected-app workflows, and multi-bot coordination. The architectural lessons transfer: keep the model transport behind a contract, make delegated work durable, separate the bot identity from the execution surface, and put a default-deny boundary in front of remote access.
The project is Apache-2.0 except for the enterprise/ directory, which has a separate source-available license. Ubuntu support is still beta, Wayland host control is disabled, and the Windows installer is not code-signed. These are not footnotes to hide. They determine whether the present build is suitable for a given deployment. For a team willing to supervise an early local-first system and contribute driver or safety improvements, the repository offers an unusually complete view of the hard parts that sit between a model demo and a multi-agent control surface.