A Routing Bug Exposed the Limit of Direct Access

ZippyDB stores product metadata, counters, and configuration for Meta, serving billions of operations per second across globally distributed clusters. The difficulty was not only request volume, but also the number and ownership of its clients. ZippyDB clients ran on more than a million hosts and were maintained by hundreds of teams. A single client could reach tens of thousands of shards spread across hundreds of thousands of database hosts, creating a dense many-to-many TLS connection graph.

That topology turned idle connections into failure amplifiers. Every connection consumed memory, CPU, and a file descriptor at both ends, while reconnects could sharply increase resource demand. Meta describes an incident in which a routing bug caused clients to open a separate connection for each shard, exhausting file descriptors and memory until the fleet entered a reboot loop. The problem was difficult to fix centrally because the clients belonged to hundreds of teams and could not be upgraded in unison.

ZGateway therefore did not begin as an attempt to make database access simply faster. It began as a way to place an otherwise uncontrollable client ecosystem behind a boundary that the platform team could operate and change.

The Two-Hop Topology Changes More Than Connection Counts

ZGateway sits between client applications and the ZServer database fleet. Instead of connecting directly to every database host it needs, a client uses a sticky connection to a regional gateway host. The gateway terminates TLS, authorizes the request against the use case ACL, applies tenant-level admission control and shaping, resolves the target shard, and forwards the request through an embedded, full-featured ZippyDB client that selects replicas. Responses are demultiplexed back to clients, with per-use-case metrics, traces, and quota usage recorded along the way.

The important change is that the backend no longer sees every client as a direct connection source. It sees a set of proxy nodes managed by the platform team. Requests from different clients targeting the same shard can be combined and batched at the gateway. Cache tiers can serve hot reads locally and use change-data-capture events to maintain a bounded-staleness contract. The proxy is therefore more than a forwarding layer; it becomes a centralized control plane for database traffic.

In Meta’s illustrative model, with 20 regions, 500,000 database hosts, 30,000 proxy hosts, and 1,000,000 clients, connections per host fell by roughly 97% to 98%, while total persistent connections fell by about 19 times. The more important result is not any single ratio. The scaling relationship changes: backend fan-in is driven mainly by regional placement and shard density on proxy hosts rather than growing linearly with the number of clients.

The Proxy Tier Starts Acting Like a Database Platform

Once traffic is centralized, capabilities that previously had to be distributed across client libraries gain a common enforcement point. ZGateway supports gradual migration by service and shard prefix, including percentage-based rollout, regional filters, and a global kill switch. For overload protection, its Discriminant Load Shedding mechanism assigns requests to tenant- and priority-specific buckets that drain in round-robin fashion, preventing one noisy tenant from consuming all shared capacity.

In a controlled overload test with roughly 1,350 tenant buckets and CPU above 90%, Meta reports that only six noisy tenants were shed. The remaining requests achieved 99.9% execution with zero rejections, while goodput stayed around 97% to 98%. The mechanism itself consumed about 8% of CPU. Centralized control can therefore isolate noisy neighbors, but it is not free: the gateway needs additional compute, memory, and scheduling logic.

The same position also handles load balancing across heterogeneous machines, cross-region failover, and transaction orchestration. Meta says client-side transaction bookkeeping was moved into the gateway and consolidated into nine phases covering all transaction traffic without a reliability regression. The word “stateless” should not be read as “without runtime state.” Caches, batching, transaction coordination, and observability still consume resources; they are simply no longer maintained independently by every client.

The Architecture’s Boundary Lies Beyond Its Headline Numbers

Meta says ZGateway now handles more than one billion operations per second, carries about 40% of ZippyDB traffic, and is projected to exceed 60%, with roughly 6% computational overhead for an average use case. These figures should not be read as an independently measured peak benchmark for the gateway. The available material also describes ZippyDB as serving billions of operations per second and does not provide latency, proxy-fleet size, or capacity measurements under a common methodology. “More than one billion per second” is therefore better treated as a deployment-scale indicator than as a directly comparable performance number.

The proxy tier also introduces explicit costs. Requests take an additional network hop. Cached reads accept a consistency boundary defined by CDC and bounded staleness. Cross-region failover requires confidence that routing, capacity, and failure domains behave as intended. The material does not disclose the actual staleness window, p99 latency impact, or validation details for regional failover, so connection reduction and throughput figures alone cannot establish that every workload will benefit equally.

For technical leaders, the reusable lesson is not simply to put a proxy in front of a database. It is to recognize when client count, ownership, and upgrade cadence have made distributed governance ineffective, and then move fan-in control, admission policies, and fault isolation into an operated layer. If the system is small, clients can be upgraded together, or minimum latency dominates every other concern, the extra hop may not be justifi