Cloudflare cuts DNS build delay in half with targeted planning

Cloudflare has significantly optimized its DNS propagation pipeline, reducing the time required to push changes to its global network by half. The engineering team achieved this by replacing broad, resource-intensive full zone rebuilds with a new targeted build planner that identifies the smallest safe unit of work for each change. This architectural shift also reduced peak memory consumption by 85%, allowing the system to handle zones with tens of millions of records more efficiently.

What happened

The core challenge stemmed from the growing scale of DNS zones on Cloudflare’s network. While the volume of events increased from 250 to roughly 700 per second since 2022, the primary bottleneck was not throughput but the computational cost of processing large zones. Some zones now contain over 20 million records, a number expected to reach hundreds of millions. Previously, any change to a zone setting or record often triggered a full build, which reads every record in the database, converts it to edge format, and compares it against the existing state. For massive zones, this process is prohibitively expensive and often impossible to fit into memory.

To solve this, Cloudflare engineers developed a targeted build planner and a unified scheduler. The planner analyzes incoming events to determine exactly which parts of a zone have changed, such as specific DNS names or shared metadata, rather than rebuilding the entire zone. The scheduler then combines multiple pending events for the same zone into a single execution window. This approach ensures that zone setting changes no longer trigger full rebuilds for zones with tens of millions of records. The result is a 50% reduction in the time from a DNS change to its data being written to Quicksilver, Cloudflare’s distributed key-value store, and an 85% drop in peak memory usage.

How it works

The optimization relies on distinguishing between different types of DNS changes and mapping them to specific build scopes. The system classifies builds into five categories: resources only (for standard record changes like A or CNAME), resources and metadata (for nameserver or wildcard changes), apex and metadata (for settings like Universal SSL), purge (for unpublished zones), and full zone (when safety cannot be proven). The planner ensures that any targeted build produces output byte-for-byte identical to what a full build would generate. If the system cannot prove that a smaller build is safe, it defaults to a full build, though for extremely large zones, it avoids full builds entirely to prevent memory exhaustion.

A critical component of this system is handling shared metadata. In Cloudflare’s architecture, multiple internal zone objects can exist for the same public domain name during lifecycle events like onboarding. A shared metadata key in Quicksilver acts as a directory, telling authoritative servers which internal zone IDs are active. When a change affects this metadata, such as a nameserver update, the builder must refresh the entire shared key. To ensure consistency, the system performs a fresh read from database replicas immediately before execution. This "plan twice" approach accounts for any database changes that occurred between the initial event and the actual build, preventing stale or incorrect data from propagating to the edge.

Key details

• Peak memory usage dropped by 85% due to the elimination of unnecessary full zone reads for large datasets.
• DNS propagation latency was halved, reducing the time from change to Quicksilver write by 50%.
• The system handles zones with over 20 million records, with expectations of growth toward hundreds of millions.
• Five build types are used: resources only, resources and metadata, apex and metadata, purge, and full zone.
• A unified scheduler combines multiple events for the same zone into a single build to improve efficiency.
• Fresh database reads occur immediately before execution to ensure the build reflects the current state.

Why it matters

For infrastructure engineers and platform teams, this case study highlights the importance of incremental processing in high-scale systems. As data sets grow, linear operations like full rebuilds become unsustainable. The shift from a "rebuild everything" model to a "rebuild only what changed" model demonstrates how precise dependency tracking can unlock significant performance gains. This is particularly relevant for teams managing large-scale configuration stores or stateful services where consistency and speed are both critical. The ability to prove safety while minimizing work allows systems to scale without proportional increases in compute resources.

The approach also offers lessons in handling eventual consistency and race conditions. By reading fresh state from replicas right before execution, Cloudflare mitigates the risk of acting on outdated information without blocking the ingestion pipeline. This pattern of conservative planning followed by last-minute verification is a robust strategy for distributed systems where event ordering and database replication lag are inherent challenges. It shows that resilience does not always require adding wait times; sometimes, it requires smarter validation at the point of action.

What you can do

• Audit your data pipelines to identify processes that perform full rebuilds when only partial updates are needed.
• Implement a classification system for changes to determine the minimum scope of work required for each event type.
• Use a unified scheduler to batch related events, reducing overhead from repeated context switching and setup.
• Verify final state against a fresh source of truth immediately before execution to handle replication lag safely.
• Set hard limits on memory-intensive operations and define fallback strategies, such as partial rebuilds, for extreme cases.
• Ensure that any targeted update produces output identical to a full rebuild to maintain data integrity guarantees.