CVE-2023-7018: Huggingface Transformers Deserialization Flaw

CVE-2023-7018 Overview

CVE-2023-7018 is an insecure deserialization vulnerability affecting the Huggingface Transformers library prior to version 4.36. The vulnerability exists in the handling of untrusted data through Python's pickle.load functionality, which can be exploited to execute arbitrary code when processing maliciously crafted model files or data.

Critical Impact

Attackers can achieve arbitrary code execution on systems processing untrusted serialized data through the Huggingface Transformers library, potentially leading to complete system compromise.

Affected Products

• Huggingface Transformers versions prior to 4.36
• Systems using transfo_xl tokenization components
• Systems using RAG (Retrieval-Augmented Generation) retrieval components

Discovery Timeline

• 2023-12-20 - CVE-2023-7018 published to NVD
• 2024-11-21 - Last updated in NVD database

Technical Details for CVE-2023-7018

Vulnerability Analysis

This vulnerability is classified as CWE-502 (Deserialization of Untrusted Data). The Huggingface Transformers library uses Python's pickle module to deserialize model data and tokenizer configurations. Prior to version 4.36, the library did not adequately restrict the use of pickle.load when processing data from potentially untrusted sources.

Python's pickle module is inherently insecure when handling untrusted data because it can deserialize arbitrary Python objects, including those that execute code during the deserialization process. An attacker can craft a malicious pickle payload that, when deserialized, executes arbitrary Python code with the privileges of the running process.

The vulnerability specifically affects the tokenization components in transfo_xl (deprecated) and the RAG retrieval module, where pickle-based deserialization is used to load cached data and model configurations.

Root Cause

The root cause stems from the unrestricted use of pickle.load in multiple modules within the Transformers library. The library did not implement proper trust boundaries or security checks before deserializing data that could originate from untrusted sources, such as remotely downloaded model files or user-supplied data.

Attack Vector

The attack requires local access and user interaction. An attacker must convince a user to load or process a maliciously crafted model file, tokenizer configuration, or serialized data through the Transformers library. This could occur through:

• Distributing malicious pre-trained models on model hubs
• Compromising existing model repositories
• Social engineering users to load malicious local files
• Man-in-the-middle attacks during model downloads

# Security patch in tokenization_transfo_xl.py
# Source: https://github.com/huggingface/transformers/commit/1d63b0ec361e7a38f1339385e8a5a855085532ce
     is_torch_available,
     logging,
     requires_backends,
+    strtobool,
     torch_only_method,
 )

The patch introduces the strtobool utility to implement trust validation, requiring explicit TRUST_REMOTE_CODE=True before allowing pickle.load operations.

# Security patch in retrieval_rag.py
# Source: https://github.com/huggingface/transformers/commit/1d63b0ec361e7a38f1339385e8a5a855085532ce
 
 from ...tokenization_utils import PreTrainedTokenizer
 from ...tokenization_utils_base import BatchEncoding
-from ...utils import cached_file, is_datasets_available, is_faiss_available, logging, requires_backends
+from ...utils import cached_file, is_datasets_available, is_faiss_available, logging, requires_backends, strtobool
 from .configuration_rag import RagConfig
 from .tokenization_rag import RagTokenizer

Detection Methods for CVE-2023-7018

Indicators of Compromise

• Unexpected process spawning from Python processes running Transformers workloads
• Unusual network connections initiated during model loading operations
• Suspicious file system modifications during deserialization operations
• Anomalous system calls during pickle.load execution

Detection Strategies

• Monitor for pickle.load calls on files from untrusted sources in Python applications
• Implement application-level logging for model loading operations
• Use static analysis tools to identify unsafe pickle deserialization patterns
• Deploy runtime application self-protection (RASP) to detect deserialization attacks

Monitoring Recommendations

• Enable verbose logging for Transformers library operations
• Monitor network traffic for connections to unauthorized model repositories
• Track file access patterns during model loading and tokenization
• Implement integrity verification for downloaded model files using cryptographic hashes

How to Mitigate CVE-2023-7018

Immediate Actions Required

• Upgrade Huggingface Transformers to version 4.36 or later immediately
• Audit existing deployments for use of deprecated transfo_xl components
• Review any RAG implementations using the affected retrieval module
• Verify the integrity and source of all pre-trained models in use

Patch Information

Huggingface has addressed this vulnerability in commit 1d63b0ec361e7a38f1339385e8a5a855085532ce. The fix introduces a trust verification mechanism that disallows pickle.load operations unless the TRUST_REMOTE_CODE environment variable is explicitly set to True. Additional details are available in the Huntr Bug Bounty Report.

Workarounds

• Only load models from trusted, verified sources until patching is complete
• Set restrictive permissions on directories containing model files
• Implement network isolation for systems processing untrusted model data
• Consider using model formats that don't rely on pickle serialization (e.g., SafeTensors)

# Configuration example - Upgrade to patched version
pip install --upgrade transformers>=4.36

# Verify installed version
pip show transformers | grep Version

# Alternative: Use SafeTensors format for model loading
export HF_HUB_ENABLE_HF_TRANSFER=1
export SAFETENSORS_FAST_GPU=1