CVE-2023-2356: Lfprojects MLflow Path Traversal Flaw

CVE-2023-2356 Overview

CVE-2023-2356 is a Relative Path Traversal vulnerability affecting MLflow, the popular open-source machine learning platform maintained by LF Projects. This vulnerability exists in versions of MLflow prior to 2.3.1 and allows remote attackers to access sensitive files on the server by manipulating path parameters with relative path sequences such as ../.

MLflow is widely used in machine learning operations (MLOps) pipelines for experiment tracking, model versioning, and deployment. The path traversal flaw enables attackers to break out of intended directory boundaries and read arbitrary files from the underlying file system, potentially exposing sensitive configuration files, credentials, and proprietary data.

Critical Impact

Remote attackers can exploit this path traversal vulnerability to read sensitive files from the MLflow server without authentication, potentially exposing machine learning models, experiment data, and system credentials.

Affected Products

• LF Projects MLflow versions prior to 2.3.1
• MLflow deployments accepting user-supplied source paths via HTTP/HTTPS, mlflow-artifacts, or local file scheme URIs
• MLflow Tracking Server instances exposed to untrusted networks

Discovery Timeline

• April 28, 2023 - CVE-2023-2356 published to NVD
• November 21, 2024 - Last updated in NVD database

Technical Details for CVE-2023-2356

Vulnerability Analysis

This path traversal vulnerability stems from insufficient validation of user-supplied source paths in MLflow's server handlers. The vulnerability allows attackers to provide URIs containing relative path components (e.g., ../) that traverse outside the intended directory structure, enabling unauthorized file access on the server.

The flaw affects multiple URI schemes supported by MLflow, including http://, https://, mlflow-artifacts://, file://, and object store URIs like s3://. When these paths contain relative traversal sequences, the server resolves them without proper sanitization, allowing access to arbitrary locations on the file system.

Root Cause

The root cause lies in the absence of path normalization and validation checks on source parameters before processing. MLflow's server handlers accepted source URIs without verifying that the resolved path remained within authorized boundaries. This allowed specially crafted paths containing ../ sequences to escape the intended directory scope.

Attack Vector

The attack can be executed remotely over the network without any authentication or user interaction. An attacker sends HTTP requests to the MLflow server containing malicious source paths with relative path traversal sequences. Example malicious paths include:

• mlflow-artifacts://host:port/../../../../etc/passwd
• http://host:port/api/2.0/mlflow-artifacts/artifacts/../../../../sensitive/data
• file://path/to/../../../../some/where/you/should/not/be

When the server processes these requests, it resolves the path and inadvertently provides access to files outside the intended artifact storage directories.

# Security patch in mlflow/server/handlers.py - Disable ability to provide relative paths in sources (#8281)
     return _wrap_response(DeleteRegisteredModelTag.Response())
 
 
+def _validate_non_local_source_contains_relative_paths(source: str):
+    """
+    Validation check to ensure that sources that are provided that conform to the schemes:
+    http, https, or mlflow-artifacts do not contain relative path designations that are intended
+    to access local file system paths on the tracking server.
+
+    Example paths that this validation function is intended to find and raise an Exception if
+    passed:
+    "mlflow-artifacts://host:port/../../../../"
+    "http://host:port/api/2.0/mlflow-artifacts/artifacts/../../../../"
+    "https://host:port/api/2.0/mlflow-artifacts/artifacts/../../../../"
+    "/models/artifacts/../../../"
+    "s3:/my_bucket/models/path/../../other/path"
+    "file://path/to/../../../../some/where/you/should/not/be"
+    """
+    source_path = urllib.parse.urlparse(source).path
+    resolved_source = pathlib.Path(source_path).resolve().as_posix()
+    # NB: drive split is specifically for Windows since WindowsPath.resolve() will append the
+    # drive path of the pwd to a given path. We don't care about the drive here, though.
+    _, resolved_path = os.path.splitdrive(resolved_source)
+
+    if resolved_path != source_path:
+        raise MlflowException(
+            f"Invalid model version source: '{source}'. If supplying a source as an http, https, "
+            "local file path, ftp, objectstore, or mlflow-artifacts uri, an absolute path must be "
+            "provided without relative path references present. Please provide an absolute path.",
+            INVALID_PARAMETER_VALUE,

Source: GitHub Commit f73147496e05c09a8b83d95fb4f1bf86696c6342

Detection Methods for CVE-2023-2356

Indicators of Compromise

• HTTP request logs containing source parameters with ../ sequences or URL-encoded variants (%2e%2e%2f)
• Unusual file access patterns in MLflow server logs indicating attempts to read files outside artifact directories
• Requests to MLflow API endpoints with mlflow-artifacts://, file://, or other URI schemes containing path traversal patterns

Detection Strategies

• Implement web application firewall (WAF) rules to detect and block requests containing path traversal patterns in URI parameters
• Monitor MLflow server access logs for suspicious source path parameters using regex patterns matching \.\.\/ or \.\.\\
• Deploy intrusion detection signatures targeting path traversal attempts in MLflow API requests
• Use file integrity monitoring on sensitive server directories to detect unauthorized access attempts

Monitoring Recommendations

• Enable verbose logging on MLflow Tracking Server instances to capture full request details including source parameters
• Configure SIEM alerts for patterns indicative of directory traversal exploitation attempts
• Monitor for anomalous file read operations from the MLflow server process
• Review access logs for requests from unexpected IP ranges targeting artifact and model endpoints

How to Mitigate CVE-2023-2356

Immediate Actions Required

• Upgrade MLflow to version 2.3.1 or later immediately on all affected deployments
• Restrict network access to MLflow Tracking Server to trusted IP ranges and authenticated users only
• Review server logs for evidence of exploitation attempts using path traversal patterns
• Implement network segmentation to limit potential impact if the vulnerability has been exploited

Patch Information

LF Projects has released MLflow version 2.3.1 which includes a fix for this vulnerability. The patch adds a validation function _validate_non_local_source_contains_relative_paths() that checks if the resolved path differs from the original source path, rejecting any paths containing relative traversal sequences.

The fix is available in commit f73147496e05c09a8b83d95fb4f1bf86696c6342. Organizations should upgrade to the patched version by running:

pip install --upgrade mlflow>=2.3.1

For detailed information, see the GitHub Commit Update and the Huntr Bounty Listing.

Workarounds

• Deploy a reverse proxy or WAF in front of MLflow to filter requests containing path traversal patterns before they reach the server
• Restrict MLflow server access to authenticated and authorized users only using network-level controls
• Run MLflow server processes with minimal file system permissions using a dedicated service account
• Consider containerizing MLflow deployments to limit file system access scope

# Configuration example - Restrict MLflow server network access
# Use firewall rules to limit access to trusted networks only
iptables -A INPUT -p tcp --dport 5000 -s 10.0.0.0/8 -j ACCEPT
iptables -A INPUT -p tcp --dport 5000 -j DROP

# Run MLflow with restricted permissions
useradd -r -s /bin/false mlflow-service
chown -R mlflow-service:mlflow-service /opt/mlflow/artifacts
sudo -u mlflow-service mlflow server --backend-store-uri sqlite:///mlflow.db --default-artifact-root /opt/mlflow/artifacts