CVE-2021-3711: OpenSSL SM2 Buffer Overflow Vulnerability

CVE-2021-3711 Overview

CVE-2021-3711 is a critical buffer overflow vulnerability in OpenSSL's SM2 decryption implementation. The flaw exists in the EVP_PKEY_decrypt() API function, which is used to decrypt SM2 encrypted data. Applications typically call this function twice—first to determine the required buffer size and then to perform the actual decryption. A bug in the buffer size calculation causes the first call to return a size that may be smaller than what the second call actually requires, leading to a heap-based buffer overflow of up to 62 bytes when processing malicious SM2-encrypted content.

Critical Impact

Attackers who can present SM2 content for decryption to a vulnerable application can trigger a heap-based buffer overflow, potentially achieving remote code execution, altering application behavior, or causing denial of service through application crashes.

Affected Products

• OpenSSL versions 1.1.1 through 1.1.1k
• Debian Linux 10.0 and 11.0
• NetApp Active IQ Unified Manager, Clustered Data ONTAP, E-Series SANtricity OS Controller, HCI Management Node, OnCommand Insight, SnapCenter, SolidFire, and related products
• Oracle Communications Cloud Native Core, MySQL Server, PeopleSoft Enterprise PeopleTools, JD Edwards EnterpriseOne Tools, and related products
• Tenable Nessus Network Monitor and Tenable.sc

Discovery Timeline

• 2021-08-24 - CVE-2021-3711 published to NVD
• 2024-11-21 - Last updated in NVD database

Technical Details for CVE-2021-3711

Vulnerability Analysis

This vulnerability resides in OpenSSL's implementation of the SM2 cryptographic algorithm, which is a Chinese national cryptographic standard. The EVP_PKEY_decrypt() function follows a two-call pattern that is common in cryptographic APIs: the first call with a NULL output buffer returns the required buffer size, and the second call performs the actual decryption into the allocated buffer.

The core issue is a miscalculation in the buffer size estimation logic. When an application queries for the required plaintext buffer size during the first call, the SM2 decryption code returns an estimate that can be smaller than the actual plaintext size produced during decryption. This discrepancy means that when the application allocates a buffer based on this erroneous size and calls EVP_PKEY_decrypt() a second time, the decryption operation can write beyond the allocated buffer boundaries.

Root Cause

The root cause is an improper calculation of the plaintext buffer size in the SM2 decryption routines. The estimation algorithm fails to account for certain padding or encoding scenarios in SM2-encrypted data, resulting in an underestimation of up to 62 bytes. This is classified as CWE-120 (Buffer Copy without Checking Size of Input), a classic buffer overflow condition where the destination buffer is insufficiently sized for the data being written.

Attack Vector

The attack vector is network-based and requires no authentication or user interaction. An attacker must be able to supply maliciously crafted SM2-encrypted data to a vulnerable application that uses OpenSSL for decryption. The attacker controls the content that overflows the buffer, potentially allowing:

• Corruption of adjacent heap metadata or application data
• Modification of program control flow leading to arbitrary code execution
• Application crashes causing denial of service

The overflow destination is application-dependent but typically occurs in heap-allocated memory, making exploitation dependent on the target application's memory layout and heap implementation.

Detection Methods for CVE-2021-3711

Indicators of Compromise

• Unexpected application crashes in processes using OpenSSL for SM2 decryption operations
• Memory corruption errors or heap integrity violations in application logs
• Anomalous network traffic containing malformed SM2-encrypted payloads targeting vulnerable services
• Signs of heap spray or memory manipulation in process memory

Detection Strategies

• Deploy intrusion detection rules to identify malformed SM2 encryption structures in network traffic
• Monitor for abnormal process termination or segmentation faults in OpenSSL-dependent applications
• Implement application-level logging around EVP_PKEY_decrypt() calls to detect unusual decryption failures
• Use memory protection tools (ASAN, Valgrind) in development environments to detect buffer overflows

Monitoring Recommendations

• Conduct inventory assessment to identify all systems running OpenSSL versions 1.1.1 through 1.1.1k
• Monitor security mailing lists and vendor advisories for related exploitation activity
• Review application logs for cryptographic operation failures that could indicate exploitation attempts
• Implement runtime memory protection mechanisms where available

How to Mitigate CVE-2021-3711

Immediate Actions Required

• Upgrade OpenSSL to version 1.1.1l or later, which contains the fix for this vulnerability
• Identify and prioritize patching of internet-facing services that process SM2-encrypted data
• Apply vendor-specific patches for affected products including Debian, NetApp, Oracle, and Tenable systems
• If SM2 is not required, consider disabling SM2 cipher support as a temporary measure

Patch Information

OpenSSL has released version 1.1.1l which addresses this vulnerability. The fix corrects the buffer size calculation in the SM2 decryption implementation to accurately estimate the required plaintext buffer size. Organizations should obtain the patched version from the OpenSSL Security Advisory 2021-08-24 or through their operating system's package manager. Additional vendor-specific patches are available from Oracle CPU October 2021 Security Alert, Oracle CPU January 2022 Security Alert, Oracle CPU April 2022 Security Alert, NetApp Advisory NTAP-20210827-0010, and Debian DSA-4963 Security Announcement.

Workarounds

• Disable SM2 cipher support in OpenSSL configuration if not required by the application
• Implement network-level filtering to block untrusted SM2-encrypted content from reaching vulnerable applications
• Deploy Web Application Firewalls (WAF) with custom rules to inspect and block malicious payloads
• Use containerization or sandboxing to limit the impact of potential exploitation

# Check OpenSSL version for vulnerability status
openssl version -a

# Expected vulnerable versions: 1.1.1 through 1.1.1k
# Update to 1.1.1l or later via package manager
# Debian/Ubuntu:
sudo apt update && sudo apt upgrade openssl

# RHEL/CentOS:
sudo yum update openssl