CVE-2024-10188: BerriAI/litellm DoS Vulnerability

CVE-2024-10188 Overview

CVE-2024-10188 is a denial-of-service vulnerability in BerriAI/litellm, an open-source proxy and SDK for interacting with large language model (LLM) providers. As of commit 26c03c9, the litellm Python server parses attacker-controlled input using Python's ast.literal_eval function. Unauthenticated remote users can send crafted payloads that exhaust server resources and crash the process. The issue is tracked as CWE-400: Uncontrolled Resource Consumption and was reported through the Huntr bounty program.

Critical Impact
Unauthenticated attackers can remotely crash the litellm Python server over the network, taking any LLM applications routed through it offline.

Affected Products

BerriAI/litellm at commit 26c03c9 and earlier
litellm Python server deployments exposing the proxy endpoint
Downstream applications that route LLM traffic through litellm

Discovery Timeline

2025-03-20 - CVE-2024-10188 published to the National Vulnerability Database
2026-06-17 - Last updated in NVD database

Technical Details for CVE-2024-10188

Vulnerability Analysis

The vulnerability originates in litellm's handling of user-supplied input on network-facing endpoints. The server passes untrusted strings to ast.literal_eval, a Python standard library function intended to safely evaluate literal expressions. While ast.literal_eval prevents arbitrary code execution, it does not bound parser complexity or recursion depth. Attackers can submit deeply nested or extremely large literal structures that consume excessive CPU, memory, or stack, causing the interpreter to hang or terminate.

Because litellm sits in front of paid LLM APIs, disrupting the proxy interrupts every downstream chat, embedding, and completion request. Availability loss aligns with the CVSS impact profile (A:H, with C:N and I:N).

Root Cause

The root cause is the assumption that ast.literal_eval is safe for arbitrary user input. It is safe against code execution but remains vulnerable to algorithmic and resource-exhaustion attacks. No length, depth, or timeout limits are enforced on the payload before it reaches the parser.

Attack Vector

The attack vector is network-based and requires no authentication or user interaction. An attacker submits a POST request containing a malformed or pathological literal (for example, a deeply nested tuple or list) to a litellm endpoint that funnels the string into ast.literal_eval. Parsing consumes worker resources until the process is killed or becomes unresponsive.

python

# Vulnerable pattern (conceptual)
import ast

def handle_user_input(payload: str):
    # payload comes directly from an unauthenticated HTTP request
    parsed = ast.literal_eval(payload)   # DoS: no size/depth limits
    return parsed

# Example pathological input: '(' * 100000 + ')' * 100000
# Triggers excessive parser work and can crash the Python server.

See the Huntr bounty listing and the upstream commit for the patched code path.

Detection Methods for CVE-2024-10188

Indicators of Compromise

litellm Python worker processes terminating with MemoryError, RecursionError, or SIGKILL from an OOM-killer
Sudden spikes in CPU or memory on litellm hosts correlated with inbound HTTP POST traffic
HTTP 5xx errors and connection resets from litellm endpoints during otherwise normal traffic
Requests containing unusually large payloads or deeply nested bracket/parenthesis sequences

Detection Strategies

Inspect reverse-proxy or WAF logs for POST bodies exceeding expected size thresholds for litellm routes
Alert on repeated crashes or restarts of the litellm service via process supervisors such as systemd or Kubernetes liveness probes
Correlate application logs referencing ast.literal_eval exceptions with the source IP address to identify probing attempts

Monitoring Recommendations

Track litellm process uptime, RSS memory, and CPU with an APM or metrics agent and alert on abnormal patterns
Enable request-rate and payload-size monitoring at the ingress or API gateway in front of litellm
Retain HTTP access logs with request body size fields to support post-incident forensics

How to Mitigate CVE-2024-10188

Immediate Actions Required

Upgrade litellm to a version that includes commit 21156ff or later
Place litellm behind an authenticated reverse proxy or API gateway and reject unauthenticated traffic
Enforce request body size limits (for example, 64–256 KB) at the ingress layer for all litellm endpoints
Restart any litellm workers currently exposed to the internet to clear potentially degraded state

Patch Information

The maintainers addressed the issue in the upstream repository. Users should pull the latest release from BerriAI/litellm and redeploy. Review the referenced commit to confirm the vulnerable ast.literal_eval call site is no longer reachable with untrusted input, and validate that container images or PyPI packages in use include the fix.

Workarounds

Restrict network exposure of the litellm proxy to trusted internal networks or VPN-only access
Terminate incoming connections at a WAF that enforces payload size and nesting-depth limits
Run litellm workers under a process manager with memory limits and automatic restart on crash to reduce downtime
Disable or gate any endpoints known to invoke ast.literal_eval on user input until the patch is applied

bash

# Example: enforce a 128KB body limit for litellm behind nginx
# /etc/nginx/conf.d/litellm.conf
server {
    listen 443 ssl;
    server_name litellm.example.com;

    client_max_body_size 128k;
    client_body_timeout  10s;

    location / {
        proxy_pass         http://127.0.0.1:4000;
        proxy_read_timeout 30s;
    }
}