Cloudflare today announced the completion of a major engineering overhaul, code-named 'Code Orange: Fail Small,' aimed at preventing the global outages that struck the company in late 2025.

The project, which spanned more than six months, introduces safer configuration deployment, automated rollback systems, and stronger incident communication protocols. Cloudflare executives say the changes directly address the root causes of the November 18 and December 5 outages that disrupted millions of websites.

'We knew we had to rebuild trust with our customers,' said John Graham-Cumming, Cloudflare's Chief Technology Officer. 'Fail Small isn't just a project name—it's a philosophy. We now ensure that any failure knocks out a tiny piece of the network before we let it touch the whole.'

Safer Configuration Changes

Central to the overhaul is a new internal system called Snapstone. It replaces the old method of instantly pushing configuration changes across the entire network.

Instead, Snapstone bundles changes into packages and releases them gradually, monitoring real-time health metrics at each step. If a problem is detected, the system automatically rolls back the change—often before any customer traffic is affected.

According to Cloudflare, high-risk configuration pipelines have been identified and equipped with independent validation tools. Product teams that were directly involved in the 2025 incidents now use the same health-mediated deployment methodology that Cloudflare already applies to software releases.

'Before Snapstone, applying health mediation to config changes was possible but required significant per-team effort,' explained Sarah Bird, Cloudflare's Senior Vice President of Engineering. 'Snapstone closes that gap by making progressive rollout and automatic rollback the default for all configuration deployments.'

Reducing the Impact of Failure

The project also focused on containing failures within smaller 'blast radii.' Cloudflare re-architected key systems so that a single faulty component cannot cascade across the network.

New 'break glass' procedures have been revised to ensure that emergency access to systems does not inadvertently bypass safety checks. Incident management processes now include mandatory peer reviews and automated escalation paths.

Background

The November 18, 2025 outage was triggered by a corrupted data file pushed globally without validation. The December 5 outage stemmed from a flawed control flag in Cloudflare's global configuration system. Both incidents paralyzed vast portions of the company's content delivery network.

Together, the outages affected hundreds of thousands of websites and exposed fragility in Cloudflare's rapid-deployment culture. The company promised a root-cause analysis and a comprehensive resilience program—which became 'Code Orange: Fail Small.'

'We knew we had to move fast, but also safely,' said Bird. 'The team essentially rebuilt our configuration pipeline from the ground up.'

Cloudflare also introduced measures to prevent drift and regressions over time. Automated tests now run against every configuration unit to ensure that previously fixed vulnerabilities do not reappear.

What This Means

For Cloudflare customers, the most visible change is increased uptime. Even if a future fault occurs, the network will now degrade gracefully rather than fail catastrophically.

Customers can also expect faster, more transparent communication during incidents. Cloudflare has overhauled its status page tools and now provides granular, per-service updates when disruptions occur.

'In most cases, internal configuration changes no longer reach our network instantly,' said Graham-Cumming. 'We catch problems and revert them before they affect your traffic. That's the new normal.'

Businesses that rely on Cloudflare for e-commerce, media delivery, or critical APIs should see no performance penalties from the new safety measures. Snapstone's health mediation runs transparently on the existing infrastructure.

Cloudflare says the effort has also fostered a cultural shift. 'Resilience is no longer a post-mortem afterthought,' Bird noted. 'It's baked into every code review and deployment decision.'

The 'Fail Small' project remains an ongoing priority. While this phase is complete, incremental improvements will continue in future quarterly cycles.

• Key milestones: Snapstone rollout, revised incident playbooks, validated high-risk pipelines
• Customer impact: Fewer and shorter outages, automatic rollbacks, better communication
• Technical debt: All past outage-causing configuration patterns have been systematically blocked

Analysts say the move positions Cloudflare to maintain its leadership in secure web performance. 'Cloudflare has turned a crisis into a competitive advantage,' said Rene Miller, a network infrastructure analyst at TechVision Research. 'Their response sets a new bar for how internet infrastructure companies should handle reliability.'