When AI Outgrows Git: Pando, Scion, and Two Ways to Rebuild Source Control

3. OpenAI Pando: keep Git as the interface, replace the server

OpenAI first examined conventional distributed Git designs. One replicated complete repositories and used three-phase commit to coordinate reference updates. It could offer every replica a consistent view, but each push had to wait for multiple rounds of communication with a majority of replicas. Disk and network latency entered the push path, and the replicas had to absorb both incoming packs and read traffic.

A second design wrote packs to object storage and ordered updates through a shared manifest acting as a write-ahead log. Pack uploads could happen in parallel, but every publication still competed to compare-and-swap the same global manifest. Even pushes to unrelated branches serialized at that point. Batching reduced some overhead by queuing updates, but increased latency. More object-store capacity did not remove the coordination bottleneck, and replicas still had to read and index incoming packs before serving them.

Pando's answer is not to make global coordination faster. It is to make coordination narrower. Updates to different branches proceed independently. Only simultaneous attempts to move the same branch conflict. A reader of one reference observes the history of the writer that produced that reference, a property called causal consistency. A client reading several branches may see them at different points in time, but a read of one branch does not jump outside its causal chain.

flowchart LR
    C["Developer or agent
git push"] --> P["Pando
streaming validation"]
    P --> B["Object storage
immutable packfile"]
    P --> M["Metadata store
ref version"]
    M --> R["Target ref
compare-and-swap"]
    F["Another client
git fetch"] --> N["Map have and want
to stored packs"]
    N --> E["Express
regional cache"]
    E --> C2["Client downloads
packfile URIs in parallel"]
    B --> E

4. How Pando parallelizes writes and streams reads

Store the pushed pack instead of repacking for every reader

A conventional Git server receives a push, verifies it, and updates a repository. When a client later fetches, the server computes another pack based on the objects each side has. Pando tries not to do that work again. It validates the incoming pack in a single streaming pass, stores the original bytes in object storage, and records a new reference version in metadata. That version points to the new tip, the negotiation dependencies, and the pack that supplied it.

A later fetch does not unpack and recompress the repository. Pando maps the client's existing and wanted commits to reference versions, then selects the historical packs needed to cover the missing objects. A pack may contain a few objects the client already has. In return, the server can serve the same bytes to many clients with little computation. This is an explicit trade: a small amount of client bandwidth for a large amount of server-side reuse.

Advertise only a small baseline so packs remain reusable

Before a push, a Git server normally advertises many refs. The more objects the client knows the server already has, the smaller it can make its pack. But with millions of branches, the advertisement alone can exceed 100 MiB. A server also needs an expensive, current view of every ref. More importantly, a pack generated against many branches has complex dependencies, and Pando cannot easily prove that the next reader has all of them.

Pando advertises a designated trunk branch by default. The pack has one clear baseline, so any client with that trunk version can fetch a feature branch using the same pack. The tradeoff is that repeated pushes to one branch, or stacked branches, resend objects outside trunk. OpenAI considers that acceptable for its workload. The median pack generated this way was under 20 KiB, and the 99th percentile was under 3 MiB.

Return an empty inline pack and download the rest in parallel

Standard Git fetch over HTTP normally permits one pack in the response. Pando uses packfile-uri, a less visible feature of the Git protocol, to return an empty inline pack plus links to multiple independent packs. The client downloads and indexes those packs in parallel instead of waiting for a server to construct one large response. OpenAI contributed improvements upstream for parallel downloads, cleanup after interrupted transfers, resumable downloads, validation, tracing, and the concept of explicit haves.

Express, the pack delivery service, caches object storage in each region. Pando uses consistent hashing to route a pack to a stable server, and cache hit rates exceed 99%. On standard cloud virtual machines, clone and fetch can saturate 10 Gbps network links. OpenAI reports more than 57 TB of Git packs served in a single hour, with more than 99% of those bytes coming from local disk cache. The system can deliver nearly a petabyte of Git data per day with only a handful of CPU cores.

Compact packs like a log-structured merge tree

If every push left a separate pack, a client that fell many days behind trunk would need tens of thousands of packs. Pando compacts them in the background using a design similar to a log-structured merge tree. Each push creates an L0 pack. Workers group consecutive reference versions into larger covers and build levels up to L4. The covers improve compression, and their indexes act like a skip list so fetch planning can run in logarithmic time relative to the distance from the reader's starting point.

Unlike many Git servers, Pando keeps packs at every level rather than discarding old ones. Different readers start at different points in history. An older L0 pack may be cheaper for reading one arbitrary object because it is less compressed. A larger higher-level pack may be better for a client catching up on a long branch. This makes the storage design serve both recently written data and older historical reads instead of optimizing only the latest commit.

5. Pando's engineering and migration results

OpenAI built Pando in Rust without legacy dependencies, moving from kickoff to production in ten weeks. One engineer designed the core, teams of agents implemented it, and a small group of engineers deployed it. The team wrote conformance tests against standard Git behavior, deterministic simulations, fuzzing, and formal models in Lean and TLA+ to expose race conditions in negotiation and pack planning. The project demonstrates how agents can accelerate infrastructure work, but the design constraints, risk decisions, and final accountability remained human responsibilities.

Deployment did not require a single cutover. Pando began as a read/write-through proxy to the existing source-control system. Both systems stayed synchronized while traffic moved in phases. Because the proxy could deduplicate and combine requests before forwarding them, each migrated workload could make the old backend healthier. OpenAI also migrated hundreds of integrations that depended on legacy APIs, webhooks, and repository downloads, and built migration skills so Codex could update developer environments and services.

In production, Pando reduced median push latency by nearly 80%, roughly a fivefold improvement, and improved merge reliability. It began serving production traffic in April 2026. OpenAI does not claim that Git itself failed. It argues that retaining the protocol and client let engineers and agents adopt the new backend without changing how they work, while allowing improvements to flow back to the open-source ecosystem.

6. Anthropic Scion: stop treating Git as the starting point

Anthropic's diagnosis is more radical. Git's strength is that every clone is a complete, self-contained repository that can compute diffs, inspect history and blame, and check out any revision offline. That remains the right design for most repositories. Anthropic's monorepo no longer matches the assumptions behind it. Most clients are always-online agents and jobs, so carrying a complete repository is a burden rather than an advantage.

At peak, Anthropic operated several hundred internal Git mirrors alongside shallow clones, sparse checkouts, promisor remotes, and repository snapshots in blob storage. These bought time but created operational complexity. Mirrors consumed disk, memory, and coordination capacity. Snapshots drifted away from the tip, and operations that needed history, such as diff and blame, required increasingly careful handling on partial clones.

The team adopted five principles. Clients are always online. Client and server are deliberately asymmetric. Source-control queries should be a service available to other tools. Every component must scale horizontally. Commits, rebases, and merges must be cheap, incremental, and free of global locks. A working copy becomes optional, sparse, and materialized on demand. Anthropic is explicit that these principles apply only to enormous monorepos with agent-dense writes. They are not evidence that ordinary Git users need to replace Git.

7. Scion's write model: a changeset is a write-ahead-log entry

Scion's unit of work is a changeset. Like a Git commit, it records author, message, timestamp, and parent information. Unlike a Git commit, it does not point to a complete root tree. A changeset lists the files that changed relative to its first parent and stores content-addressed pointers to compressed file contents in object storage. Changesets are created on the server and become visible everywhere immediately.

This structure supports high write throughput. The server does not accept a packfile, take a global repository lock, or coordinate every server before publishing a commit. Scion technically supports refs, but creating and fetching commits does not depend on them. Moving a ref is a compare-and-swap on one Spanner row. The global state that every server must agree on is restricted to a small number of main and release references.

A changeset contains only a delta, so it cannot answer questions such as who last changed a line or which commits affected a path. Scion recursively builds additional structures, including file blame, path history, commit-graph skip lists, copy detection indexes, and path search indexes. It calls these structures derived data. Each type is computed from the corresponding values of the changeset's parents. A data type can be computed eagerly or deferred, and clients never see its internal representation, so the server can replace an index without changing the client protocol.

flowchart LR
    W["Client edits files
and submits a change"] --> S["Scion server
creates changeset"]
    S --> O["Content-addressed storage
compressed file objects"]
    S --> X["Spanner
changeset and ref metadata"]
    X --> D["Recursive derived data
blame, path history, graph indexes"]
    C["Checkout client"] --> G["Request missing trees and blobs"]
    G --> DN["Node daemon"]
    DN --> RC["Regional cache"]
    RC --> SV["Authoritative Scion server"]
    O --> SV
    X --> SV

8. Scion's read model: ask what is missing, not what everyone already has

Git fetch is a negotiation. The client tells the server which commits it has and which commits it wants, and the server computes the difference between the reachable object sets before constructing a pack. Scion changes the question. The client starts from a root tree, checks its local store for missing trees, requests them recursively, and then requests the corresponding blobs. Every object is content-addressed and immutable, so any cache can answer without coordinating with another cache.

This has three direct consequences. Requests are naturally resumable because there is no multi-gigabyte pack response. Sparse and even virtualized checkouts need no special protocol because the client simply does not request trees it does not want. The server and caches do not have to recompute deltas or perform delta compression for each client. The architecture looks more like a conventional tiered object cache than a Git server.

The Scion client talks over a local socket to scion-daemon, which maintains objects shared by all jobs on that node. The daemon starts from a periodically refreshed snapshot and requests tree and blob objects from a regional cache, scion-cache. The regional cache streams objects directly from memory in the disk format the daemon consumes. On a miss, it asks authoritative scion-server processes, which read from Spanner or cloud storage as needed.

Each tier maintains a cache hit rate above 99% and avoids long-lived state, which makes scaling straightforward. Anthropic deliberately enforces the hierarchy to isolate failures. A regional cache outage must not send every request upstream and collapse the authoritative servers. The Scion infrastructure footprint is about 200 times smaller than the previous Git mirroring system in CPU and memory. CI and research checkouts fell from minutes to seconds, roughly two orders of magnitude faster, and CI alone saved 25% of CPU capacity.

9. Scion's gains and its costs

A seven-person team built Scion, the Antique code-review surface, code search, a source-control IAM system, and several related systems, then fully migrated the monorepo in five months. During the same period, the number of changes merged per day grew tenfold. Anthropic argues that rapid development by small teams will become more common as coding agents improve.

On quality, the team built hermetic simulations of infrastructure interactions, fuzzing, conformance tests, and rate limiting. Claude found an algorithmic optimization for ancestry queries that had missed an index, work that might otherwise consume days or weeks of expert analysis. Engineers still review every change to the new system, but they increasingly review intent, system boundaries, and constraints instead of reading every line. The article gives examples where agents tried to linearly scan history to satisfy a local pattern, or added excessive validation in response to a hypothetical risk. Engineers had to tell them no.

Scion also changes merge policy. A conventional merge queue often imposes a global repository lock and tests a series of changes in order. At agent scale, hundreds or thousands of changes may await that queue, making the queue itself the bottleneck. Anthropic could not effectively reproduce the global lock, so it invested in post-merge and pre-deploy testing. Concurrent changes can conflict, but Claude can often repair the conflict faster than a human can identify it. Whether that tradeoff is safe depends on rollback, test coverage, and production risk, not merge speed alone.

Finally, Anthropic says Scion is not a suitable Git replacement for general use. Adopting it would mean replacing review, search, permissions, CI, and every surrounding integration, while spending far more on internal model usage than most organizations would consider practical. The team hopes to contribute the underlying ideas to Git and forge ecosystems rather than package its bespoke system as a universal product.

10. The real difference between the two routes

Compatibility limits the migration blast radius

Pando's most underrated advantage is not throughput. It is the size of the migration boundary. Standard Git clients continue to work, so engineers do not change commands and many editors, review tools, and CI systems remain usable. OpenAI can start with a transparent proxy and migrate hundreds of integrations one at a time. At every stage, the old backend can remain a fallback, and failures can be isolated to a workload rather than the whole development lifecycle.

Scion gains more freedom but accepts a much larger migration. Without Git compatibility, clients, code review, search, permissions, CI, and automation interfaces all need to be rebuilt around the new protocol. Anthropic completed the work in five months with seven engineers and extensive agent support, while spending an unusual amount on internal tokens and infrastructure. Another company may borrow one data-structure idea without being able to copy the migration plan.

Causal consistency is not the same thing as an unreliable branch

Pando does not provide a point-in-time snapshot of every branch, but updates to one reference remain serializable. A client can read its own write and can follow the dependencies from trunk to a feature branch. Development workflows usually care about the order within a reference, not whether they observed every one of tens of thousands of refs at the same microsecond. Tools that assume one query can return a globally consistent repository view must be redesigned.

Scion moves the consistency problem into changesets and server-side queries. Appending a changeset does not move a global ref. Only a small number of main and release refs require compare-and-swap. The system does not impose one total order on every commit. Instead, it computes derived data on demand and exposes history through service APIs that can scale independently.

Problem	Pando	Scion
Many unrelated branches push at once	Coordinate each ref independently; only the target ref contends	Append changesets to a write-ahead log without a global ref lock
Many jobs read the same history	Reuse pushed packs and serve them through regional caches	Fetch only missing trees and blobs from immutable object caches
History and blame queries	Keep the Git commit graph while reducing repacking	Recursively generate blame, path history, and graph indexes
Cross-branch snapshot	Give it up to parallelize writes	Do not rely on one repository snapshot; combine server-side query results
Offline client capability	Retain Git's local objects and negotiation	Reduce the client to a bounded online cache

Compaction is an architecture choice, not a storage footnote

Pando turns a packfile from a temporary response into a reusable distribution unit. Background compaction solves the problem of too many historical packs while retaining multiple levels for readers at different distances from the tip. Scion removes packfiles altogether and lets object storage and cache tiers serve immutable trees and file content. One preserves Git's object semantics; the other replaces negotiation and packing with object requests.

That is why Scion is not merely a faster Git server. It changes what the client knows, what it asks for, and what it caches. It also changes who creates commits and merges. Pando remains Git at its core, but changes the server from a system that recomputes each response to one that records and reuses the history produced by writes.

11. What the evidence proves and what it does not

Both companies report production results, but the boundaries matter. First, these are internal workloads, not one normalized cross-company benchmark. Pando's nearly 80% reduction in median push latency and Scion's two-orders-of-magnitude checkout improvement cannot be transferred directly to a smaller organization. Repository size, branch count, file distribution, network location, cache warmth, and agent behavior all change the result.

Second, Pando is still an OpenAI internal production system. Its public Git improvements and patches can be adopted, but the complete backend is not presented as a general product. Scion is even more explicit about its narrow fit. Its review, search, and CI systems are bespoke. The articles provide architectural evidence and operational lessons, not an open-source distribution that another company can install and expect to reproduce the same outcomes.

Third, neither article publishes complete cost accounting. Scion's infrastructure is 200 times smaller in CPU and memory, but the post does not provide a full old-versus-new bill, dual-running cost during migration, or total engineering spend. Pando describes nearly a petabyte per day but does not translate object-store writes, compaction, cache storage, and cross-region traffic into one total cost. These figures are directions, not purchasing quotes.

Fourth, permissions, security, and auditability remain separate design problems. Anthropic describes repository and directory-level access control plus semantic constraints in review. OpenAI emphasizes secure software foundations and conformance tests against standard Git behavior. Rebuilding source control is not only performance engineering. Every new cache, proxy, and server-side query must preserve identity, authorization, and audit boundaries.

12. A practical decision ladder

If Git is only occasionally slow, start by fixing CI, branch policy, sparse checkout, caching, and monorepo boundaries. Moving jobs away from full clones and avoiding duplicate fetches is usually far cheaper than building a source-control system. Build caching, test selection, and workspace reuse can remove load that was never intrinsic to Git.

If the bottleneck is clearly inside the Git server and the organization must keep its current tools, Pando's principles are the more practical place to start. Preserve reference semantics, limit coordination to one reference, reuse immutable packs, advertise a deliberate baseline, support parallel pack downloads, and make compaction observable. These can be introduced through managed services, proxy layers, and caches before anyone rewrites the complete backend.

Scion becomes a reasonable option only at Anthropic's scale or when an organization is prepared to migrate the whole development lifecycle. The threshold is not a 10 GB repository. It is millions of checkouts per day, tens of thousands of merges, hundreds of agents and jobs writing at once, and a Git workflow that can no longer be stabilized through ordinary engineering. Even then, the organization must decide who owns permissions, review, CI, rollback, offline development, and training.

Current symptom	First action	When to change architecture
Occasional slow fetches and repeated CI downloads	Dependency caching, sparse checkout, build caches, deduplicated jobs	When caching no longer covers the dominant traffic
Pushes queue as branches grow, but one branch remains healthy	Revisit repository boundaries or per-reference server coordination	When unrelated branch contention becomes a persistent failure
Agents and jobs repeatedly read the same history	Immutable pack or object caches, regional delivery, parallel downloads	When the server still builds a separate response for each client
Tens of thousands of daily merges and agent-dense writes	Rethink review, merge queues, CI, and source-control boundaries together	When preserving Git requires replacing the whole development lifecycle

13. The dividing line is the workload assumption

Taken together, the most useful lesson is that neither company treated "Git is slow" as a single point failure. OpenAI found enough value in client compatibility to rebuild the server. Anthropic found that always-online clients contradicted Git's complete-copy assumptions and rebuilt the system. Both decisions began with observed workload, not with a desire to build a new database.

For robotics, embodied AI, and physical AI teams, the same reasoning applies to other large assets. Training logs, simulation scenes, sensor data, and model weights can create one class of repository pressure. Agent-generated code, evaluation scripts, and experiment configuration create another. Do not put everything into one repository and discover the architecture problem later. Separate immutable large objects, versioned code, reproducible datasets, and temporary artifacts, then choose storage and cache boundaries for each.

Pando is the incremental route: accept Git as the ecosystem interface and replace the most expensive server-side paths. Scion is the extreme route: when agent volume and merge velocity exceed the old workflow, replace the interface, client, and CI together. For most organizations, the useful next step is neither copy. Measure push contention, repeated reads, checkout traffic, and merge latency first. Then decide whether the bottleneck is inside Git or in everything built around it.

In one sentence: Pando makes Git faster under its existing contract, while Scion shows what source control looks like when that contract becomes the bottleneck. Pando is the more incremental engineering path; Scion is an architecture experiment that only extreme load and exceptional investment can support.

---

Sources: OpenAI, "Scaling Git at OpenAI," published November 2, 2026 (openai.com/index/scaling-git-at-openai); Anthropic, "Redesigning source control at Anthropic," published October 13, 2026, updated October 15, 2026 (anthropic.com/research/redesigning-source-control). This bilingual article is a complete translation and adaptation of both sources, with explicitly identified RobotWorld analysis. All figures are from the corresponding original articles.