CVE-2026-24271: NVIDIA TensorRT-LLM DoS Vulnerability

CVE-2026-24271 Overview

NVIDIA TensorRT-LLM contains a resource allocation vulnerability in its OpenAI-compatible inference API. An attacker with local access can submit requests that trigger unrestricted allocation of GPU resources without any throttling or quota enforcement. Successful exploitation exhausts GPU memory and compute capacity, producing a denial-of-service condition that disrupts inference workloads sharing the affected host.

The flaw is categorized under CWE-770 (Allocation of Resources Without Limits or Throttling). Because TensorRT-LLM is widely deployed in production AI serving stacks, an outage in this component can halt downstream large language model (LLM) applications and multi-tenant inference services.

Critical Impact
A local attacker can exhaust GPU resources through the OpenAI-compatible inference API, causing denial of service for all workloads on the affected accelerator.

Affected Products

NVIDIA TensorRT-LLM (OpenAI-compatible inference API component)
Deployments exposing the TensorRT-LLM inference endpoint to local users or co-located tenants
AI serving stacks and containers built on vulnerable TensorRT-LLM releases

Discovery Timeline

2026-07-14 - CVE-2026-24271 published to the National Vulnerability Database
2026-07-15 - Last updated in NVD database

Technical Details for CVE-2026-24271

Vulnerability Analysis

TensorRT-LLM exposes an OpenAI-compatible HTTP API so clients can send inference requests using familiar chat and completion schemas. The affected endpoint accepts these requests and allocates GPU memory, KV cache slots, and compute streams to service them. The implementation does not enforce per-client or per-request limits on how much GPU resource a single caller can consume.

An attacker who can reach the API locally submits requests crafted to maximize allocation. These may include very long context windows, large max_tokens values, oversized batch sizes, or high-concurrency streaming sessions. The server accepts and queues each request, pinning GPU memory and scheduler slots until the device is saturated.

Once the GPU is exhausted, legitimate inference requests fail with allocation errors or hang indefinitely. Because modern GPUs are shared across model replicas and tenants, the impact extends to every workload bound to the affected device.

Root Cause

The root cause is missing resource governance in the OpenAI-compatible API path. The service lacks quotas, rate limiting, and per-request caps that would bound GPU memory and compute usage. CWE-770 classifies this pattern as allocation of resources without limits or throttling.

Attack Vector

Exploitation requires local access to the inference endpoint and does not require authentication or user interaction. The attack targets availability only; confidentiality and integrity of model data are not affected. The vulnerability manifests entirely through legitimate-looking API calls, so no exploit binary or memory corruption primitive is required. Refer to the NIST CVE-2026-24271 Record for the authoritative technical description.

Detection Methods for CVE-2026-24271

Indicators of Compromise

Sudden and sustained spikes in GPU memory utilization or KV cache occupancy reported by nvidia-smi or DCGM exporters.
Inference API latency growth followed by allocation failures such as CUDA out of memory or scheduler timeouts.
Repeated requests from a single local client with abnormally large max_tokens, prompt length, or concurrent stream counts.

Detection Strategies

Baseline normal GPU utilization per model replica and alert on statistical deviations sustained beyond a short window.
Log every request to the OpenAI-compatible endpoint with client identity, prompt length, and requested token budget, then hunt for outliers.
Correlate inference service error logs with GPU telemetry to identify when allocation failures coincide with a specific caller.

Monitoring Recommendations

Ingest NVIDIA DCGM and container runtime metrics into a centralized analytics platform for cross-host correlation.
Monitor process-level GPU handles to detect single processes holding disproportionate memory.
Track API request volume and payload sizes per source address to surface abuse patterns early.

How to Mitigate CVE-2026-24271

Immediate Actions Required

Restrict network exposure of the TensorRT-LLM OpenAI-compatible API to trusted local processes only, using host firewalls or Unix socket bindings.
Place an authenticating reverse proxy in front of the endpoint and enforce per-client rate limits and maximum request size.
Cap max_tokens, context length, and concurrent request counts at the proxy or gateway layer until a vendor patch is applied.

Patch Information

NVIDIA has not published a fixed version reference within the NVD record at the time of writing. Monitor the CVE.org CVE-2026-24271 Details page and NVIDIA security bulletins for the official patched release, and upgrade all TensorRT-LLM deployments once the fix is available.

Workarounds

Run TensorRT-LLM inside a container with GPU memory limits enforced through the NVIDIA container runtime --gpus constraints and cgroup quotas.
Deploy an API gateway that enforces token bucket rate limiting and rejects oversized payloads before they reach the inference server.
Isolate multi-tenant inference workloads on dedicated GPUs so that resource exhaustion by one tenant cannot cascade to others.

bash

# Configuration example: enforce request size and rate limits at an NGINX reverse proxy
http {
    limit_req_zone $binary_remote_addr zone=trtllm:10m rate=5r/s;
    client_max_body_size 32k;

    server {
        listen 127.0.0.1:8443 ssl;
        location /v1/ {
            limit_req zone=trtllm burst=10 nodelay;
            proxy_pass http://127.0.0.1:8000;
            proxy_read_timeout 30s;
        }
    }
}