Rewterz Threat Update – Zero-Day Vulnerability ‘HTTP/2 Rapid Reset’ Exploited to Launch Record-Breaking DDoS Attacks

Severity

High

Analysis Summary

A new distributed denial of service (DDoS) attack technique known as ‘HTTP/2 Rapid Reset’ has been actively exploited as a zero-day vulnerability since August. This technique has shattered previous records in terms of the scale of the attacks. Major internet infrastructure providers, including Amazon Web Services, Cloudflare, and Google, have come together to report their efforts in mitigating these attacks. The vulnerability exploited in these attacks has been tracked as CVE-2023-44487, carrying a CVSS score of 7.5 out of 10, signifying its severity.

Amazon reported mitigating attacks reaching 155 million requests per second, Cloudflare handled 201 million rps, and Google astonishingly faced an attack of 398 million rps. Google managed to thwart these attacks by enhancing the capacity of their network’s edge.

The DDoS attacks leveraged the HTTP/2 protocol’s multiplexing capabilities, which allow multiple requests to be transmitted over a single TCP connection as concurrent streams. One key feature of HTTP/2 is the ability for a client to terminate a request by issuing a RST_STREAM frame. The HTTP/2 Rapid Reset attack utilized this feature to send and rapidly cancel requests in succession. By doing so, it bypassed the server’s concurrent stream limit and overloaded the server without reaching its configured threshold.

“HTTP/2 rapid reset attacks consist of multiple HTTP/2 connections with requests and resets in rapid succession,” mentioned AWS.

In practical terms, these attacks involve multiple HTTP/2 connections with rapid sequences of requests and resets. Each request generates logs before being reset or canceled by the client. This tactic allows for an indefinite number of requests in flight on each connection, enabling attackers to inundate a target website, making it unresponsive to new incoming requests. What’s alarming is that these attacks can be executed using a relatively modestly-sized botnet, around 20,000 machines.

“This zero-day provided threat actors with a critical new tool in their Swiss Army knife of vulnerabilities to exploit and attack their victims at a magnitude that has never been seen before,” mentioned Cloudflare.

The severity of these attacks is underscored by the fact that they provided threat actors with a powerful tool to exploit and attack victims on an unprecedented scale. These attacks are not only highly effective but also difficult to defend against due to their exploitation of a fundamental protocol.

HTTP/2 is widely used on the internet, with approximately 35.6% of all websites employing it. Consequently, the implications of these attacks are significant, as they can potentially impact a large portion of the internet. Google Cloud reported observing various variants of the Rapid Reset attacks, some of which were more efficient than the standard HTTP/2 DDoS attacks.

Each of the three companies agrees that the optimal strategy for clients to defend against HTTP/2 Rapid Reset attacks involves deploying all accessible HTTP-flood protection mechanisms and reinforcing their DDoS resilience through a combination of multifaceted mitigation techniques.

F5, an independent advisory, recommended that its customers update their NGINX configuration to limit the number of concurrent streams and persist HTTP connections for a specific number of requests, in order to mitigate the impact of this attack. 

In response to the public disclosure of CVE-2023-44487, numerous companies, including Alibaba Tengine, Apache Tomcat, Apple Swift, Eclipse Jetty, F5, Golang, Kubernetes, and major Linux distributions like Debian, Red Hat, and Ubuntu, as well as Microsoft and Netty, have released updates to counter this new attack vector in their software. 

It is emphasized that organizations need to take proactive measures to ensure protection against these types of attacks, as threat actors are now likely to be aware of this HTTP/2 vulnerability, which may lead to an arms race between defenders and attackers in terms of patching and exploitation. 

Impact

• Disruption of Online Services

Indicators Of Compromise

CVE

• CVE-2023-44487

Affected Vendors

F5
Google
Amazon
HAProxy
Envoy
Eclipse
nghttp2
Alibaba
Caddy
gRPC
hyperium

Affected Products

• Google Cloud Platform
• Amazon Web Services
• Project Envoy 1.27.0
• HAProxy 2.8.0
• HAProxy 2.9-dev7
• HAProxy 2.9-dev6
• HAProxy 2.9-dev5
• Jetty 12.0.1
• Jetty 11.0.16
• nghttp2 1.56.0
• nghttp2 1.55.1
• nghttp2 1.55.0
• nghttp2 1.54.0
• Alibaba Group Tengine 3.0.0
• Alibaba Group Tengine 2.4.1a
• Alibaba Group Tengine 2.4.0
• hyperium hyper 1.0.0-rc.4
• hyperium hyper 0.14.27
• hyperium hyper 0.14.26
• Caddy 2.7.4
• Caddy 2.7.3
• Caddy 2.7.2
• Caddy 2.7.1
• gRPC-Go 1.58.2
• gRPC-Go 1.58.0
• gRPC-Go 1.58.1

Remediation

• Refer to the Vendor’s Patches Links for patch, upgrade or suggested workaround information including Google Cloud Platform, Amazon Web Services, HAProxy, F5 NGINX, Envoy Project, Eclipse, nghttp2, Alibaba, Hyperium, Caddy, and gRPC 
• Implement rate limiting for incoming requests to limit the number of requests from a single source within a specified time frame. This can help mitigate the impact of rapid request and reset attacks.
• Implement thorough request validation to filter out malicious or unnecessary requests. This can help reduce the volume of requests that need to be processed and minimize the impact of the attack.
• Deploy IDPS solutions to detect and block abnormal traffic patterns associated with DDoS attacks. These systems can identify and respond to suspicious behavior in real-time.
• A WAF can filter and monitor incoming traffic to an application and block or allow traffic based on a defined set of security rules. Configure your WAF to detect and block suspicious HTTP/2 Rapid Reset attack patterns.
• Keep all software, including web servers and application frameworks, up to date with the latest security patches. 
• Continuously monitor network traffic and establish baselines for normal activity.
• Use load balancers to distribute incoming traffic across multiple servers. Load balancers can help prevent a single server from being overwhelmed by an attack.
• Implement multi-factor authentication (MFA) for administrative access to critical systems and infrastructure to prevent unauthorized access during attacks.
• Implement robust monitoring and logging solutions to capture detailed data on network and application activity. This information can be invaluable for post-attack analysis and forensics.

Also, here are some of the recommendations mentioned in Cloudflare blog:

• Thoroughly analyze your external and partner network’s external connections. This assessment will help identify Internet-facing systems that may be vulnerable. Implement necessary mitigations promptly.
• Evaluate your existing security measures and capabilities designed to protect, detect, and respond to attacks. Ensure that these defenses are up to date and address any identified issues in your network promptly.
• Place your DDoS protection measures outside your data center. This strategic placement is essential because, once malicious traffic reaches your data center, mitigating a DDoS attack becomes more challenging.
• Employ comprehensive DDoS protection strategies, including protection at the application (Layer 7) level. Implement Web Application Firewalls (WAFs) to safeguard against application-specific attacks. 
• Regularly update and maintain the security of your web servers and operating systems, especially those facing the Internet. Additionally, ensure that all automation processes, such as Terraform builds and image deployments, are fully patched to prevent the accidental use of outdated and vulnerable versions.
• Consider Protocol Downgrades: As a last resort, consider temporarily disabling HTTP/2 and HTTP/3 protocols. This is a drastic step because it may significantly impact performance. Use this measure if all other defenses prove insufficient.