CVE-2024-21802: Llama.cpp Buffer Overflow Vulnerability

CVE-2024-21802 Overview

A heap-based buffer overflow vulnerability exists in the GGUF library info->ne functionality of llama.cpp Commit 18c2e17. This vulnerability allows attackers to achieve code execution through specially crafted .gguf files. When a user opens or processes a malicious GGUF model file, the vulnerability can be triggered, potentially leading to complete system compromise.

Critical Impact

This vulnerability enables remote code execution through malicious GGUF model files, requiring no privileges and no user interaction beyond loading the file, posing significant risk to systems using llama.cpp for AI model inference.

Affected Products

• ggerganov llama.cpp (Commit 18c2e17 and related versions)

Discovery Timeline

• 2024-02-26 - CVE CVE-2024-21802 published to NVD
• 2025-11-04 - Last updated in NVD database

Technical Details for CVE-2024-21802

Vulnerability Analysis

This vulnerability represents a heap-based buffer overflow (CWE-122, CWE-787) in the GGUF library's info->ne functionality within llama.cpp. The GGUF format is used for storing and loading Large Language Model (LLM) weights and metadata. The vulnerability occurs during the parsing of GGUF files when the library processes tensor dimension information without proper bounds checking.

When llama.cpp reads a GGUF model file, it parses metadata including tensor information stored in the ne (number of elements per dimension) array. A maliciously crafted GGUF file can specify dimension values that cause the parser to allocate an insufficiently sized buffer, which is subsequently overflowed during data population. This heap corruption can be leveraged for arbitrary code execution.

Root Cause

The root cause is insufficient validation of tensor dimension parameters when parsing GGUF files. The info->ne functionality does not properly verify that the dimension values specified in the file are within acceptable bounds before allocating memory and writing data. This allows an attacker to control the size of heap allocations and the amount of data written to them, creating the classic heap overflow condition.

Attack Vector

The attack vector is network-based, as malicious GGUF files can be distributed through various channels including model repositories, direct downloads, or social engineering. The exploitation scenario involves:

1. An attacker crafts a malicious .gguf file with manipulated tensor dimension values
2. The victim downloads or receives the malicious model file
3. When the victim loads the model using llama.cpp for inference, the vulnerability is triggered
4. The heap overflow corrupts adjacent memory structures
5. Through careful heap manipulation, the attacker achieves arbitrary code execution

The vulnerability is particularly concerning in AI/ML workflows where users frequently download and experiment with third-party models from community repositories. For additional technical details, see the Talos Intelligence Vulnerability Report.

Detection Methods for CVE-2024-21802

Indicators of Compromise

• Unexpected crashes or memory corruption errors when loading GGUF model files
• Process execution anomalies following model file loading operations
• Unusual network connections or child processes spawned by llama.cpp applications
• Abnormally sized or structured .gguf files with suspicious tensor dimension metadata

Detection Strategies

• Implement file integrity monitoring for GGUF model files in use
• Monitor process behavior of applications using llama.cpp for unexpected memory access patterns
• Deploy endpoint detection solutions capable of identifying heap exploitation techniques
• Utilize application-level monitoring to detect anomalous model loading behavior

Monitoring Recommendations

• Enable crash dump collection and analysis for applications using llama.cpp
• Implement logging for all GGUF file load operations including source and file hashes
• Monitor for suspicious file downloads with .gguf extension from untrusted sources
• Deploy memory protection mechanisms such as heap canaries and ASLR where available

How to Mitigate CVE-2024-21802

Immediate Actions Required

• Update llama.cpp to the latest version that includes the security patch
• Audit all GGUF model files currently in use and verify their source integrity
• Only load GGUF files from trusted, verified sources
• Implement network segmentation to limit exposure of systems running llama.cpp

Patch Information

Organizations should update llama.cpp to the latest available version from the official ggerganov repository. The vulnerability was identified in Commit 18c2e17, and subsequent commits have addressed this heap overflow issue. Review the project's commit history and release notes for specific patch information. The Talos Intelligence advisory provides additional details on affected versions.

Workarounds

• Avoid loading GGUF files from untrusted or unverified sources until patching is complete
• Implement sandboxing or containerization for llama.cpp processes to limit the impact of potential exploitation
• Use application whitelisting to restrict model files to known-good checksums
• Consider running llama.cpp processes with reduced privileges and restricted file system access

# Configuration example - Run llama.cpp in a sandboxed environment
# Using firejail as an example sandbox solution
firejail --private --net=none --caps.drop=all ./main -m trusted_model.gguf

# Alternatively, use Docker with restricted capabilities
docker run --rm --read-only --cap-drop=ALL \
  -v /path/to/trusted/models:/models:ro \
  llama-cpp-image ./main -m /models/model.gguf