OpenAI has built Pando, a Git-compatible backend designed to scale Git in the company’s monorepo by relaxing repository-wide consistency and reusing packfiles. The system replaces the service layer beneath the Git protocol while keeping the Git client and developer workflows intact. According to OpenAI, Pando now powers development in its monorepo and yields substantial improvements in push latency, read throughput, and merge reliability.
Pando was created to meet two primary engineering goals: enable many engineers and agent-assisted tools to push changes concurrently to different branches, and deliver read throughput that can saturate client network links. To achieve those aims without forcing users to change how they interact with Git, OpenAI preserved compatibility with the standard Git client and protocol. Developers continue to use familiar commands and integrations while Pando handles scale beneath the surface.
The core of Pando’s design departs from conventional approaches that enforce a single global order for repository updates. Instead, Pando records a sequence of ref versions per branch and stores incoming packfiles as immutable blobs in a scalable blob store. A metadata store tracks ref versions, their dependencies on packfiles, and the current ref pointers. This approach guarantees serializability for individual refs—so operations against the same branch behave as expected—while providing a weaker, causal consistency model across the repository as a whole. By relaxing repository-wide ordering, Pando enables unrelated pushes to proceed in parallel and scale horizontally as more servers and storage are added.
Writes and reads are handled very differently from a traditional Git server. When a client pushes, Pando streams the packfile directly into blob storage and performs a single-pass validation of the pack without unpacking it into a local repository. A multi-key transaction then records a new ref version and atomically updates the branch pointer, giving the pushing client read-your-own-write consistency. Because metadata writes remain constant in number regardless of packfile size, pushes to unrelated refs do not bottleneck on pack size and can be executed concurrently.
For fetches, Pando maps a client’s declared “have” and “want” commitments to recorded ref versions. Each ref version references previously stored packfiles; the server selects a collection of existing packs that cover the objects the client needs. Rather than producing a bespoke packfile for each request, Pando serves reusable packfile URIs and relies on a delivery service called Express that caches packs regionally. This reuse reduces CPU and disk I/O on origin servers and allows fetches to approach the full capacity of client network links.
To prevent a proliferation of tiny packfile downloads, Pando runs background compaction workers that merge small initial packs (L0) into progressively larger covers. The compaction scheme is similar to log-structured merge-tree techniques and improves compression while enabling fetch planning in logarithmic time relative to how far a client is behind the repository tip. OpenAI runs Pando clusters with local caches for both packfiles and metadata, and uses consistent hashing to route requests to Express servers. The company reports cache hit rates above 99%.
In production, Pando moves significant volume: nearly a petabyte of Git data per day and more than 57 terabytes of Git packs served in a single hour, with over 99% of those bytes coming from local disk cache. OpenAI says that Pando reduced median push latency in the monorepo by nearly fivefold and improved merge reliability—concrete operational gains that accompany the scalability improvements.
Engineering rigor underpinned Pando’s development. The implementation is written in Rust, and OpenAI applied conformance tests against standard Git behavior, deterministic simulation harnesses to surface race conditions, and formal-specification tooling using systems such as Lean and TLA+ to reason about correctness. The team reports moving from project start to production in ten weeks, with Pando beginning to serve production traffic in April.
Initial rollouts used Pando as a read- and write-through proxy on top of OpenAI’s existing source-control backend to enable a safe, zero-downtime migration. OpenAI distributed a public Git build to developer machines and the ChatGPT desktop app and used an internal wrapper around the Git CLI to phase the transition. The company also migrated integrations that relied on legacy APIs and webhooks, and used Codex to assist teams in preparing checkouts and environments for the switchover.
Working with the broader Git ecosystem, OpenAI contributed fixes and enhancements back to upstream Git. These included improvements for parallel pack downloads, validation of existing packs during fetches, tracing for packfile-URI downloads, and an explicit-haves concept that allows servers to advertise branch tips separately from histories a client might omit. Some patches have been merged upstream and others are available in OpenAI’s public Git repository.
Pando is intended to scale OpenAI’s developer workflows as agents and engineers produce more parallel work, and to support scenarios in which many agents collaborate on shared repositories. OpenAI says it is exploring ways to share the technology more broadly while continuing to work with the open-source Git community. By rethinking the trade-offs between consistency, reuse, and operational efficiency, Pando demonstrates a pragmatic path to scale Git for very large, highly concurrent monorepos.
In summary, Pando represents a targeted engineering effort to scale Git in a large monorepo: it preserves Git compatibility, relaxes global ordering to enable concurrency, reuses packfiles to reduce server load, and relies on compaction and caching to keep fetch performance high. The result is a system that supports thousands of parallel pushes and high-throughput reads while integrating with standard Git tooling and contributing improvements back to the ecosystem.
Source: Read the original source
Leave a Reply