CVE-2024-12704: Llamaindex DOS Vulnerability

CVE-2024-12704 Overview

CVE-2024-12704 is a denial of service vulnerability in the LangChainLLM class of the run-llama/llama_index repository, version v0.12.5. The stream_complete method executes the LLM using a thread and retrieves results via the get_response_gen method of the StreamingGeneratorCallbackHandler class. When the thread terminates abnormally before _llm.predict executes, the code lacks exception handling for this condition. This causes an infinite loop [CWE-835] inside get_response_gen, exhausting CPU resources. An attacker can trigger the condition by supplying input of an incorrect type.

Critical Impact
Remote, unauthenticated attackers can cause a persistent infinite loop in LlamaIndex worker processes, exhausting compute resources and disrupting availability of downstream LLM-powered applications.

Affected Products

LlamaIndex llama_index version 0.12.5
Applications embedding the LangChainLLM class from llama-index-core
Deployments using StreamingGeneratorCallbackHandler for streaming completions

Discovery Timeline

2025-03-20 - CVE-2024-12704 published to NVD
2026-06-17 - Last updated in NVD database

Technical Details for CVE-2024-12704

Vulnerability Analysis

The flaw resides in llama-index-core/llama_index/core/langchain_helpers/streaming.py. The stream_complete method dispatches the underlying language model call to a worker thread. That thread is expected to push tokens into a queue consumed by get_response_gen, which yields them to the caller.

The consumer loop in get_response_gen blocks on the queue and continues until it observes a termination signal from the worker. When the worker thread raises before reaching _llm.predict, no sentinel or exception is ever placed on the queue. The consumer therefore waits forever, and the process spins without producing a response.

Because the trigger is malformed input rather than a crafted payload, any user-controlled parameter that reaches the LLM call surface is a viable attack path. Applications exposing LlamaIndex over an HTTP API compound the impact by pinning worker resources per abusive request.

Root Cause

The root cause is missing exception handling around the threaded LLM invocation combined with an unbounded wait in the streaming generator. The producer thread can die silently, but the consumer has no timeout, no error channel, and no health check to detect the dead producer.

Attack Vector

An unauthenticated attacker sends a request that reaches stream_complete with an input whose type does not match what the underlying LangChain LLM expects. The worker thread raises a TypeError before _llm.predict runs. The generator then loops indefinitely, holding CPU and worker resources.

python

# Patch excerpt from llama-index-core/llama_index/core/langchain_helpers/streaming.py
+import time
 from queue import Queue
 from threading import Event
 from typing import Any, Generator, List, Optional

Source: GitHub commit d1ecfb7. The fix introduces a time-based timeout so the callback handler no longer blocks forever when the producer thread dies.

Detection Methods for CVE-2024-12704

Indicators of Compromise

Sustained 100% CPU usage in Python worker processes hosting LlamaIndex without corresponding output generation
Long-lived HTTP requests to endpoints backed by LangChainLLM.stream_complete that never emit a response body
Repeated client requests carrying malformed or non-string prompt parameters to LlamaIndex-backed APIs

Detection Strategies

Instrument LlamaIndex request handlers with per-request wall-clock timers and alert when streaming completions exceed expected latency ceilings
Compare thread lifetimes in the LlamaIndex worker pool against completed response counts to surface stuck generator threads
Log and inspect TypeError and other unhandled exceptions originating from LangChain LLM adapter code paths

Monitoring Recommendations

Track process-level CPU and thread counts for services importing llama_index.core.langchain_helpers.streaming
Emit metrics from the streaming endpoint for queue depth, tokens produced per second, and request duration
Alert on outbound HTTP requests to model providers that terminate without a downstream response returning to the client

How to Mitigate CVE-2024-12704

Immediate Actions Required

Upgrade llama-index-core to a version containing commit d1ecfb7 or later, which adds a timeout to the LangChain callback handler
Validate and coerce all user-controlled inputs to the expected string type before passing them to stream_complete
Place LlamaIndex streaming endpoints behind a reverse proxy that enforces request timeouts and concurrency limits

Patch Information

The fix is committed in run-llama/llama_index commit d1ecfb7, titled fix: add a timeout to langchain callback handler (#17296). The patch imports time and applies a bounded wait in StreamingGeneratorCallbackHandler so a dead producer thread no longer causes an infinite loop. Additional context is available in the Huntr bug bounty listing.

Workarounds

Wrap calls to stream_complete in a caller-side timeout using concurrent.futures or asyncio.wait_for to bound generator execution
Add strict input validation at the API boundary to reject non-string or malformed prompt payloads before they reach the LLM adapter
Restrict access to LlamaIndex streaming endpoints via authentication and rate limiting until the upgrade is deployed

bash

# Pin a patched version of llama-index-core after upgrading
pip install --upgrade 'llama-index-core>0.12.5'

# Verify the installed version no longer matches the vulnerable release
python -c "import llama_index.core, sys; print(llama_index.core.__version__)"