CVE-2025-53537: Oisf Libhtp DOS Vulnerability

CVE-2025-53537 Overview

CVE-2025-53537 is a memory leak vulnerability in LibHTP, a security-aware parser for the HTTP protocol widely used in network security monitoring solutions including Suricata IDS/IPS. The vulnerability exists in versions 0.5.50 and below, where improper memory management in the LZMA decompression error handling path allows attackers to induce memory exhaustion through specially crafted network traffic.

Critical Impact

Traffic-induced memory leak can starve the monitoring process of memory, leading to loss of network visibility and potential denial of service for security monitoring infrastructure.

Affected Products

• LibHTP versions 0.5.50 and below
• Suricata IDS/IPS deployments using vulnerable LibHTP versions
• Network security appliances leveraging LibHTP for HTTP parsing

Discovery Timeline

• 2025-07-23 - CVE-2025-53537 published to NVD
• 2025-08-05 - Last updated in NVD database

Technical Details for CVE-2025-53537

Vulnerability Analysis

This vulnerability is classified as CWE-401 (Missing Release of Memory after Effective Lifetime), representing a memory leak condition in the LZMA decompression component of LibHTP. The flaw occurs during error handling in the decompression pipeline, where memory allocated for LZMA state is not properly freed when decompression fails.

LibHTP serves as a critical component in network security monitoring solutions, parsing HTTP traffic for inspection by intrusion detection and prevention systems. When processing HTTP responses with LZMA-compressed content, the library allocates memory for the decompression state. Under normal operation, this memory is properly released. However, when an error occurs during decompression, the error handling path fails to call LzmaDec_Free(), leaving the allocated memory orphaned.

An attacker can exploit this by sending malformed or specially crafted HTTP traffic that triggers repeated decompression errors. Each failed decompression attempt leaks memory, gradually exhausting available system resources. This can ultimately cause the monitoring process to crash or become unresponsive, creating a blind spot in network security monitoring.

Root Cause

The root cause lies in the error handling logic within htp/htp_decompressors.c. When the inflate operation fails and returns an error code, the code sets drec->zlib_initialized to HTP_COMPRESSION_OVER and returns HTP_ERROR without first checking if LZMA decompression was in use. If LZMA was being used, the LzmaDec_Free() function must be called to release the allocated decompression state before marking the compression as complete.

Attack Vector

The vulnerability is exploitable over the network without authentication or user interaction. An attacker can craft malicious HTTP responses with LZMA-compressed payloads designed to trigger decompression errors. By sending a sustained volume of such traffic, the attacker can progressively exhaust memory on systems running vulnerable versions of LibHTP.

The attack is particularly concerning because it targets security monitoring infrastructure. Successful exploitation could:

1. Cause the monitoring process to crash due to out-of-memory conditions
2. Degrade performance as memory pressure increases
3. Create gaps in network visibility during memory exhaustion
4. Potentially allow other malicious traffic to pass undetected

                // There is data even if there is an error
                // So use this data and log a warning
                htp_log(d->tx->connp, HTP_LOG_MARK, HTP_LOG_WARNING, 0, "GZip decompressor: inflate failed with %d", rc);
+               if (drec->zlib_initialized == HTP_COMPRESSION_LZMA) {
+                   LzmaDec_Free(&drec->state, &lzma_Alloc);
+               }
                drec->zlib_initialized = HTP_COMPRESSION_OVER;
                return HTP_ERROR;

Source: GitHub Commit 9037ea35

The patch adds a conditional check to verify if LZMA decompression was active before the error occurred. If so, it properly frees the LZMA decoder state using LzmaDec_Free() before returning the error, preventing the memory leak.

Detection Methods for CVE-2025-53537

Indicators of Compromise

• Unusual memory growth patterns in processes using LibHTP (e.g., Suricata)
• Increasing memory consumption correlated with HTTP traffic volume
• Out-of-memory errors or process crashes in network monitoring applications
• Log entries showing repeated LZMA decompression failures

Detection Strategies

• Monitor memory usage trends for LibHTP-dependent applications over time
• Implement alerting on abnormal memory growth rates in IDS/IPS processes
• Analyze HTTP traffic logs for patterns of malformed LZMA-compressed responses
• Deploy application performance monitoring to track memory allocation patterns

Monitoring Recommendations

• Configure memory usage alerts for Suricata and other LibHTP-based applications
• Implement process watchdogs to detect and restart crashed monitoring services
• Enable verbose logging for HTTP decompression errors to identify potential attacks
• Establish baseline memory consumption metrics for comparison during incidents

How to Mitigate CVE-2025-53537

Immediate Actions Required

• Upgrade LibHTP to version 0.5.51 or later which contains the security fix
• If immediate upgrade is not possible, disable LZMA decompression as a workaround
• Monitor affected systems for signs of memory exhaustion until patched
• Consider implementing process restart policies for memory-related failures

Patch Information

The vulnerability has been addressed in LibHTP version 0.5.51. The fix ensures that LzmaDec_Free() is called in the error handling path when LZMA decompression fails, properly releasing allocated memory. Organizations should update to this version or later as soon as possible.

For additional technical details, refer to the GitHub Security Advisory and the security patch commit.

Workarounds

• Disable LZMA decompression in Suricata configuration as described below
• Implement memory limits and automatic restart for affected processes
• Deploy upstream filtering to reduce exposure to malicious HTTP traffic
• Consider network segmentation to limit attacker access to monitoring infrastructure

# Suricata configuration workaround (suricata.yaml)
app-layer:
  protocols:
    http:
      libhtp:
        default-config:
          lzma-enabled: false