Scaling to 1 million concurrent sandboxes in seconds
Modal's v2 sandbox scheduler achieves 1M concurrent sandboxes at under 500ms median creation latency by removing all central datastores from the critical path — replacing serialized global coordination with a fleet of stateless scheduling servers that make decisions against async-replicated in-memory worker state and contact workers directly via two-hop RPC, explicitly trading global consistency for horizontal scalability.
Architecture and design guidance: one of the clearest real-world accounts of why centralized scheduling hits a wall and exactly how to redesign around it. The article names the specific failure mode (O(sandboxes) Postgres writes in the critical path, single serialized scheduler), the exact replacement (stateless schedulers + async Redis state propagation + direct worker RPC), the kernel-level issue they hit (rtnl lock contention when many containers start simultaneously), and the remaining bottleneck (single Redis stream, shardable when needed). The Kubernetes comparison is honest about what rewriting etcd would cost. The consistency tradeoff is stated plainly, not buried. Best engineering article of the day.