CVE-2023-43654: PyTorch TorchServe SSRF Vulnerability

CVE-2023-43654 Overview

CVE-2023-43654 is a critical Server-Side Request Forgery (SSRF) vulnerability in TorchServe, PyTorch's production model serving and scaling tool. The default configuration lacks proper input validation, enabling third parties to invoke remote HTTP download requests and write files to the disk. This vulnerability affects TorchServe versions 0.1.0 through 0.8.1 and can be exploited to compromise the integrity of the system and access sensitive data.

The vulnerability stems from TorchServe's model registration functionality, which allows users to load models from arbitrary URLs without adequate validation when using default settings. Attackers can leverage this weakness to perform SSRF attacks, potentially leading to remote code execution through deserialization of malicious model files.

Critical Impact

Unauthenticated attackers can remotely invoke HTTP download requests and write arbitrary files to disk, potentially compromising system integrity and enabling remote code execution through malicious model deserialization.

Affected Products

• PyTorch TorchServe versions 0.1.0 through 0.8.1
• Systems running TorchServe with default allowed_urls configuration
• Production ML model serving environments using unpatched TorchServe instances

Discovery Timeline

• September 28, 2023 - CVE-2023-43654 published to NVD
• November 21, 2024 - Last updated in NVD database

Technical Details for CVE-2023-43654

Vulnerability Analysis

This SSRF vulnerability exists in TorchServe's model registration mechanism due to inadequate input validation in the default configuration. The allowed_urls parameter, which controls which URLs models can be loaded from, is insufficiently restrictive by default. This allows attackers to specify arbitrary URLs when registering new models, causing the TorchServe server to make outbound HTTP requests to attacker-controlled servers and write the downloaded content to the local filesystem.

The vulnerability is classified under CWE-918 (Server-Side Request Forgery), where the application constructs a URL using user-controllable input and retrieves the contents without proper validation. This is particularly dangerous in the context of PyTorch model serving because downloaded model files undergo deserialization, which can lead to arbitrary code execution if the model file contains malicious serialized objects.

Root Cause

The root cause of CVE-2023-43654 lies in TorchServe's default configuration not properly restricting the allowed_urls parameter. When this parameter is set to its default value, TorchServe does not adequately validate or restrict the URLs from which models can be downloaded. This permissive default allows any URL to be used for model registration, creating an SSRF attack vector.

The user of TorchServe is responsible for configuring both the allowed_urls and specifying the model URL to be used. However, the lack of security warnings or restrictive defaults in versions prior to 0.8.2 meant many deployments were vulnerable out of the box.

Attack Vector

The attack vector for this vulnerability is network-based and requires no authentication or user interaction. An attacker can exploit this vulnerability by:

1. Sending a model registration request to an exposed TorchServe Management API endpoint
2. Specifying a malicious URL pointing to an attacker-controlled server hosting a weaponized model file
3. TorchServe fetches the malicious model from the attacker's server and writes it to disk
4. Upon model loading, the malicious serialized content is deserialized, potentially executing arbitrary code

The vulnerability can be chained with PyTorch's pickle deserialization to achieve remote code execution. Technical details and proof-of-concept information are available in the Packet Storm advisory.

Detection Methods for CVE-2023-43654

Indicators of Compromise

• Unexpected outbound HTTP/HTTPS requests from TorchServe server to external or internal IP addresses
• Unusual model files appearing in TorchServe's model store directory
• Model registration API requests containing external URLs or internal network addresses
• Unexpected process spawning or command execution originating from the TorchServe process

Detection Strategies

• Monitor TorchServe Management API endpoints for model registration requests with suspicious or external URLs
• Implement network traffic analysis to detect SSRF patterns such as requests to internal IP ranges, cloud metadata endpoints, or known malicious infrastructure
• Deploy Web Application Firewall (WAF) rules to inspect and block model registration requests containing unauthorized URLs
• Enable verbose logging on TorchServe to capture all model registration activities including source URLs

Monitoring Recommendations

• Configure alerting for any model registration attempts using URLs outside of approved domains
• Monitor file system changes in TorchServe's model storage directories for unexpected new files
• Track network connections originating from the TorchServe process to detect potential SSRF exploitation
• Implement runtime application self-protection (RASP) to detect and block malicious deserialization attempts

How to Mitigate CVE-2023-43654

Immediate Actions Required

• Upgrade TorchServe to version 0.8.2 or later immediately, as this release includes warnings when default allowed_urls values are used
• Configure the allowed_urls parameter to explicitly whitelist only trusted and necessary model sources
• Restrict network access to the TorchServe Management API using firewall rules or network segmentation
• Audit existing TorchServe deployments for any signs of unauthorized model registrations

Patch Information

PyTorch has addressed this vulnerability in TorchServe version 0.8.2. The fix, merged through Pull Request #2534, adds a warning to alert users when the default value for allowed_urls is being used. Users are strongly advised to upgrade to the patched version 0.8.2 or later.

For detailed information about the vulnerability and remediation steps, refer to the GitHub Security Advisory GHSA-8fxr-qfr9-p34w.

Workarounds

• There are no known workarounds for this vulnerability according to the vendor advisory
• Organizations unable to immediately upgrade should restrict Management API access to localhost only and use network segmentation to isolate TorchServe instances
• Implement strict egress filtering to prevent TorchServe from making outbound connections to unauthorized destinations
• Consider deploying TorchServe behind an API gateway that validates and restricts model registration requests

# Configuration example - Restrict allowed_urls in config.properties
# Specify only trusted model sources
allowed_urls=https://your-trusted-model-registry.internal/models

# Bind Management API to localhost only
management_address=http://127.0.0.1:8081