Evidence at a glance
Shared Containers Let Workloads Affect One Another
In an October 2026 article, OpenAI described how Habitat, its online storage platform, supports data access for products including ChatGPT, API, and Codex. Built on Azure Cosmos DB, Habitat is intended to provide fast, reliable storage access across multiple geographic regions. The article is not primarily about a database feature; it is about balancing reliability, performance, and operational effort when many products share underlying storage.
The early design placed different product workloads in shared Cosmos DB containers, but sharing does not make workloads compatible by itself. Different products may use the same logical partition keys, such as user_id or workspace_id. When traffic concentrates on a key or physical partition, one workload can consume throughput needed by another. OpenAI describes a case in which traffic from a smaller feature interfered with core login and registration flows, turning the efficiency of shared resources into a potential path for failures to spread.
Dedicated Containers Solved Isolation—and Added Work
For a time, the team assigned dedicated containers to important new workloads, limiting the impact of a failure to a particular service or feature. This made ownership and isolation boundaries clearer and reduced the chance that one tenant would directly consume a shared container’s resources. But isolation came at a cost: each container needed capacity planning and advance scaling, which product teams could miss. Workloads also lost the buffering effect of aggregated demand, potentially leaving capacity idle during quieter periods.
That reversal shows that single-tenancy and multi-tenancy solve different problems. Dedicated containers make it easier to limit the blast radius, but distribute capacity management across more teams. Shared containers simplify centralized management and pool resources, but leave workloads exposed to noisy neighbors. Habitat’s return to multi-tenancy by default did not mean abandoning isolation. It meant trying to make isolation a control that could be applied according to workload and request, rather than a permanent container boundary for every product.
Putting Isolation Into Traffic and Partition Controls
For shared containers to remain reliable, the system must distinguish between request types and their importance. OpenAI describes prioritizing throttling for the hottest keys and placing low-priority work, such as offline bulk writes and data extraction, behind interactive requests. When resources are scarce, the system can limit lower-priority traffic first to protect user-facing paths. This does not eliminate contention for shared resources; it gives the system a way to decide who gets resources first and what gets throttled first.
Another change concerns how database partitions scale. Cosmos DB distributes data by logical partition key and uses physical partitions to serve throughput. When a physical partition splits, throughput that was concentrated on it can be divided across the new partitions. OpenAI says it worked with Azure to raise the serverless limit from 5,000 to 10,000 RU/s per physical partition and add splitting based on RU consumption, rather than relying only on storage growth. For operators, the aim is to reduce the need to monitor capacity continuously and expand containers by hand. The public account does not provide a cost comparison for this change.
Global Low Latency Comes From a Combination, Not Full Replication
For cross-region access, Habitat does not replicate all data everywhere. OpenAI says it places data nearer to users where capacity allows, while a small set of highly latency-sensitive data, such as authentication metadata, is replicated across regions. Some critical accounts are replicated to all 40 Azure regions; other accounts reach at least one region on each continent. The article also notes that nearby placement is constrained by storage capacity, electricity, and cooling, so one global deployment strategy cannot apply to every dataset.
Caching covers another part of the read path. Habitat uses Valkey clusters close to request processing to cache recently accessed objects and relationships, then propagates cache invalidations through change data capture from the Cosmos DB change feed. The pipeline processes millions of changes per second and distributes them through more than 100 Kafka topics. TTLs help limit how long stale data can persist during delayed invalidation or abnormal conditions. Read performance therefore depends not just on the cache, but also on change propagation, underlying replication, and failure behavior. The materials do not report a cache hit rate or a formal maximum-staleness guarantee.
A 12x Read-Latency Improvement Is Not an End-to-End Promise
OpenAI reports that Habitat’s global read p50 fell from about 120 milliseconds to about 10 milliseconds, an improvement of roughly 12x. The company attributes the result to a combination of regional caching, cross-region replication for critical data, and a rewrite of the Habitat service from Python to Rust. It also says its largest Cosmos DB account stores tens of petabytes and serves more than 20 million QPS, while CDC spans thousands of containers and millions of container partitions. These figures indicate substantial scale, but the claim of serving more than one billion users does not specify a measurement period or deduplication method. It should not be read as one billion unique ChatGPT users, nor as a measure of concurrent load.
For technical leaders, the transferable lesson is not to copy a particular RU limit or number of replication regions. It is to map business criticality, request priority, and data latency sensitivity to resource policies: which traffic must be protected, which can be throttled, and which data merits the cost of cross-region replication. The reported 12x result is an overall read-latency figure; OpenAI does not separate the contribution of each optimization, and it is not a claim that ChatGPT’s end-to-end response time improved by 12x. The multi-tenancy approach is still evolving, and quantitative details on cache staleness, cost, and isolation rules remain undisclosed. Before adopting a similar design, teams should define service levels, tolerated staleness, and cost limits, then test whether sharing is genuinely more efficient than dedicat