CVE-2023-7018: Huggingface Transformers Deserialization Flaw

CVE-2023-7018 Overview

CVE-2023-7018 is an insecure deserialization vulnerability affecting the Huggingface Transformers library prior to version 4.36. The vulnerability exists in the handling of untrusted data through Python's pickle.load functionality, which can be exploited to execute arbitrary code when processing maliciously crafted model files or data.

Critical Impact
Attackers can achieve arbitrary code execution on systems processing untrusted serialized data through the Huggingface Transformers library, potentially leading to complete system compromise.

Affected Products

Huggingface Transformers versions prior to 4.36
Systems using transfo_xl tokenization components
Systems using RAG (Retrieval-Augmented Generation) retrieval components

Discovery Timeline

2023-12-20 - CVE-2023-7018 published to NVD
2024-11-21 - Last updated in NVD database

Technical Details for CVE-2023-7018

Vulnerability Analysis

This vulnerability is classified as CWE-502 (Deserialization of Untrusted Data). The Huggingface Transformers library uses Python's pickle module to deserialize model data and tokenizer configurations. Prior to version 4.36, the library did not adequately restrict the use of pickle.load when processing data from potentially untrusted sources.

Python's pickle module is inherently insecure when handling untrusted data because it can deserialize arbitrary Python objects, including those that execute code during the deserialization process. An attacker can craft a malicious pickle payload that, when deserialized, executes arbitrary Python code with the privileges of the running process.

The vulnerability specifically affects the tokenization components in transfo_xl (deprecated) and the RAG retrieval module, where pickle-based deserialization is used to load cached data and model configurations.

Root Cause

The root cause stems from the unrestricted use of pickle.load in multiple modules within the Transformers library. The library did not implement proper trust boundaries or security checks before deserializing data that could originate from untrusted sources, such as remotely downloaded model files or user-supplied data.

Attack Vector

The attack requires local access and user interaction. An attacker must convince a user to load or process a maliciously crafted model file, tokenizer configuration, or serialized data through the Transformers library. This could occur through:

Distributing malicious pre-trained models on model hubs
Compromising existing model repositories
Social engineering users to load malicious local files
Man-in-the-middle attacks during model downloads

python

# Security patch in tokenization_transfo_xl.py
# Source: https://github.com/huggingface/transformers/commit/1d63b0ec361e7a38f1339385e8a5a855085532ce
     is_torch_available,
     logging,
     requires_backends,
+    strtobool,
     torch_only_method,
 )

The patch introduces the strtobool utility to implement trust validation, requiring explicit TRUST_REMOTE_CODE=True before allowing pickle.load operations.

python

# Security patch in retrieval_rag.py
# Source: https://github.com/huggingface/transformers/commit/1d63b0ec361e7a38f1339385e8a5a855085532ce
 
 from ...tokenization_utils import PreTrainedTokenizer
 from ...tokenization_utils_base import BatchEncoding
-from ...utils import cached_file, is_datasets_available, is_faiss_available, logging, requires_backends
+from ...utils import cached_file, is_datasets_available, is_faiss_available, logging, requires_backends, strtobool
 from .configuration_rag import RagConfig
 from .tokenization_rag import RagTokenizer

Detection Methods for CVE-2023-7018

Indicators of Compromise

Unexpected process spawning from Python processes running Transformers workloads
Unusual network connections initiated during model loading operations
Suspicious file system modifications during deserialization operations
Anomalous system calls during pickle.load execution

Detection Strategies

Monitor for pickle.load calls on files from untrusted sources in Python applications
Implement application-level logging for model loading operations
Use static analysis tools to identify unsafe pickle deserialization patterns
Deploy runtime application self-protection (RASP) to detect deserialization attacks

Monitoring Recommendations

Enable verbose logging for Transformers library operations
Monitor network traffic for connections to unauthorized model repositories
Track file access patterns during model loading and tokenization
Implement integrity verification for downloaded model files using cryptographic hashes

How to Mitigate CVE-2023-7018

Immediate Actions Required

Upgrade Huggingface Transformers to version 4.36 or later immediately
Audit existing deployments for use of deprecated transfo_xl components
Review any RAG implementations using the affected retrieval module
Verify the integrity and source of all pre-trained models in use

Patch Information

Huggingface has addressed this vulnerability in commit 1d63b0ec361e7a38f1339385e8a5a855085532ce. The fix introduces a trust verification mechanism that disallows pickle.load operations unless the TRUST_REMOTE_CODE environment variable is explicitly set to True. Additional details are available in the Huntr Bug Bounty Report.

Workarounds

Only load models from trusted, verified sources until patching is complete
Set restrictive permissions on directories containing model files
Implement network isolation for systems processing untrusted model data
Consider using model formats that don't rely on pickle serialization (e.g., SafeTensors)

bash

# Configuration example - Upgrade to patched version
pip install --upgrade transformers>=4.36

# Verify installed version
pip show transformers | grep Version

# Alternative: Use SafeTensors format for model loading
export HF_HUB_ENABLE_HF_TRANSFER=1
export SAFETENSORS_FAST_GPU=1