Cloudflare misconfiguration behind recent BGP route leak

Cloudflare has shared more details about a recent 25-minute Border Gateway Protocol (BGP) route leak affecting IPv6 traffic, which caused measurable congestion, packet loss, and approximately 12 Gbps of dropped traffic.

The BGP system helps route data across different networks called autonomous systems (AS) that send it to destination through smaller networks on the internet.

The incident was caused by an accidental policy misconfiguration on a router and affected external networks beyond Cloudflare customers.

“During the incident on January 22, we caused a similar kind of route leak, in which we took routes from some of our peers and redistributed them in Miami to some of our peers and providers,” reads the Cloudflare announcement.

“According to the route leak definitions in RFC7908, we caused a mixture of Type 3 and Type 4 route leaks on the Internet.”

A BGP route leak occurs when an Autonomous System (AS) violates valley-free routing policies by incorrectly advertising routes learned from one peer or provider to another peer or provider.

As a result, traffic is sent through a network that was never intended to carry it. This often causes congestion, drops, or suboptimal paths. When firewall filters are used to accept traffic only from specific providers, the traffic is completely discarded.

Valley-free rules describe how routes are supposed to be propagated based on business relationships between networks, and when they are violated, traffic is attracted to networks that can’t carry it via longer or unstable paths, and, like in this case, dropped entirely.

Although such incidents primarily cause reliability issues, they do have a security dimension, as they can lead to unauthorized parties intercepting and analyzing traffic in BGP hijacking incidents.

Cloudflare explained that the root cause of the BGP route leak was a policy change intended to prevent Miami from advertising Bogotá IPv6 prefixes.

Removing specific prefix lists made the export policy overly permissive, allowing a route-type internal match to accept all internal (iBGP) IPv6 routes and export them externally.

“As a result, all IPv6 prefixes that Cloudflare redistributes internally across the backbone were accepted by this policy, and advertised to all our BGP neighbors in Miami,” explained Cloudflare.

Cloudflare detected the problem shortly after it appeared, and its engineers manually reverted the configuration and paused automation, stopping the impact within 25 minutes. The triggering code change was later reverted, and automation was safely re-enabled.

The internet giant says that this latest case is very similar to a July 2020 incident and has also listed measures to prevent such occurrences in the future.

The proposed measures include adding stricter community-based export safeguards, CI/CD checks for policy errors, improved early detection, validating RFC 9234, and promoting RPKI ASPA adoption.