OpenSSL patches high-severity DTLS memory leak in latest release

OpenSSL released security updates on September 29 to address a high-severity vulnerability in its Datagram Transport Layer Security (DTLS) implementation. The flaw, tracked as CVE-2026-84782, allows an attacker to potentially leak heap memory or cause a denial of service by triggering a specific race condition during handshake retransmissions. Fixes are publicly available for recent branches, while older versions now require paid premium support.

What happened

The vulnerability affects the DTLS protocol, which is essentially TLS adapted for UDP traffic. It is commonly used in real-time communication systems like WebRTC for voice and video calls. The issue arises when a large handshake message is being sent in fragments. If the network pauses transmission mid-message, a timer may trigger a resend of an earlier message. Due to a logic error, the resend operation incorrectly uses the buffer position of the paused large message rather than resetting to the start of the message being resent.

This mismatch causes the resent packet to contain leftover bytes from the larger message, labeled incorrectly. These bytes can include unencrypted data from the application's heap memory, exposing sensitive information to the remote peer. In worse cases, if the read operation accesses unmapped memory, the application crashes. Laurent Gaffie of Secorizon reported the issue on August 17, and Ryan Hooper developed the patch. OpenSSL has not confirmed if an attacker can reliably force this condition, nor have any exploits been observed in the wild.

The fix is included in OpenSSL 4.0.3, 3.6.5, 3.5.9, and 3.4.8. However, the situation is more complex for users of older branches. OpenSSL 3.0 reached its end of public security support on September 7. Consequently, the fix for version 3.0 (released as 3.0.23) is available only to customers with premium support contracts. The same restriction applies to the long-deprecated 1.1.1 and 1.0.2 branches.

How it works

DTLS handles unreliable UDP connections by fragmenting large handshake messages into smaller datagrams. When a connection stalls, the protocol uses a retransmission timer to resend messages that might have been lost. The bug occurs in the interaction between this timer and the fragmentation logic. When a large message is partially sent and then paused, the internal state tracks the current position in the buffer. If the timer fires for a different, earlier message while the large one is paused, the code mistakenly continues writing from the paused position instead of the start of the resent message.

This results in a packet that claims to be a handshake message but contains arbitrary data from the heap. Because the label is wrong, the receiving end may process this data as valid protocol content, leading to information leakage. If the buffer overrun hits invalid memory addresses, the process terminates abruptly. The flaw affects both DTLS clients and servers, and the fix ensures that retransmissions always reset the buffer pointer correctly.

Key details

• CVE Identifier: CVE-2026-84782, rated High severity by OpenSSL and 8.2/10 by CISA.
• Affected Protocol: DTLS only; standard TLS over TCP is not impacted.
• Public Fixes: Available in OpenSSL 4.0.3, 3.6.5, 3.5.9, and 3.4.8.
• Restricted Fixes: OpenSSL 3.0.23, 1.1.1zj, and 1.0.2zs are available only to premium support customers.
• Distribution Updates: Ubuntu 26.04, 24.04, and 22.04 have released patched packages; Debian 13 is fixed, but Debian 12 remains vulnerable as of September 30.
• Other Flaws: This release also fixes 13 other issues, including a moderate severity crash in OpenSSL 4.0 (CVE-2026-84783).

Why it matters

For developers building real-time communication tools, this vulnerability poses a direct risk to user privacy and service availability. Heap memory leaks can expose session keys, personal data, or internal application state. Since DTLS is foundational for WebRTC, any service using voice, video, or real-time data channels over UDP must verify their OpenSSL dependency. The fact that no workaround exists means updating is the only mitigation path.

The shift of OpenSSL 3.0 to premium-only security updates marks a significant change for many Linux distributions and embedded systems. Teams relying on Long Term Support (LTS) versions of Ubuntu or Debian that bundle OpenSSL 3.0 must now rely on their distribution maintainer for backports or purchase commercial support. This adds operational complexity and potential cost to maintaining secure infrastructure, highlighting the need for proactive dependency management.

What you can do

• Identify all services using OpenSSL for DTLS, particularly WebRTC gateways or VoIP servers.
• Upgrade to the latest public fixed version: 4.0.3, 3.6.5, 3.5.9, or 3.4.8.
• If using OpenSSL 3.0, apply distribution-specific patches (e.g., Ubuntu’s libssl3t64 update) and reboot if required.
• Consider migrating from OpenSSL 3.0 to a currently supported branch like 3.5 or 4.0 to ensure future public security updates.
• Monitor Debian 12 systems closely, as they remain vulnerable until a patch is released.
• Review logs for unusual crashes or handshake failures that might indicate attempted exploitation.