CVE-2026-65110: NVIDIA Megatron-LM RCE Vulnerability

CVE-2026-65110 Overview

NVIDIA Megatron-LM contains an insecure deserialization vulnerability in its checkpoint loading routine. An attacker can craft a malicious checkpoint file that executes arbitrary code when a user loads it into Megatron-LM. The flaw is classified under CWE-502: Deserialization of Untrusted Data.

Successful exploitation can lead to arbitrary code execution, privilege escalation, information disclosure, and data tampering on the host running the framework. Because machine learning practitioners routinely download and share checkpoints from public hubs and collaborators, the social-engineering requirement is low.

Critical Impact

Loading an attacker-controlled Megatron-LM checkpoint grants code execution with the privileges of the user running the training or inference workload.

Affected Products

• NVIDIA Megatron-LM (checkpoint loading component)
• Deployments consuming third-party or untrusted Megatron-LM checkpoint files
• Training and inference pipelines built on vulnerable Megatron-LM releases

Discovery Timeline

• 2026-10-13 - CVE-2026-65110 published to NVD
• 2026-10-13 - Last updated in NVD database
• Vendor advisory - Published via the NVIDIA Security Repository

Technical Details for CVE-2026-65110

Vulnerability Analysis

Megatron-LM loads model checkpoints that embed serialized Python objects. The loader reconstructs these objects without validating their type or provenance. An attacker who controls checkpoint contents can embed objects whose reconstruction triggers arbitrary code paths, including shell commands and Python callables.

The vulnerability requires local access and low privileges, but no user interaction beyond initiating a checkpoint load. Impact spans confidentiality, integrity, and availability because the resulting code runs inside the training process with full access to datasets, credentials, and GPU resources.

Machine learning workflows amplify the risk. Checkpoints are commonly shared through model hubs, collaboration repositories, and automated CI pipelines. A single poisoned artifact can compromise multiple downstream users and training clusters.

Root Cause

The root cause is unsafe deserialization of checkpoint data, consistent with CWE-502. The loader trusts the serialized object graph instead of restricting allowed classes or using a safe tensor-only format. Any constructor or __reduce__ method embedded in the checkpoint runs during load.

Attack Vector

An attacker publishes or delivers a crafted checkpoint file. A victim loads the checkpoint using the standard Megatron-LM loading API. Deserialization invokes attacker-controlled code with the privileges of the Python process, achieving local code execution without authentication to the host beyond the user's existing session.

No verified public exploit code is available at the time of writing. See the NVIDIA Security Repository and the NVD entry for CVE-2026-65110 for authoritative technical details.

Detection Methods for CVE-2026-65110

Indicators of Compromise

• Checkpoint files sourced from untrusted model hubs, forums, or email attachments prior to loading.
• Python training processes spawning unexpected child processes such as sh, bash, curl, wget, or python -c.
• Outbound network connections from GPU training hosts to unfamiliar domains shortly after a checkpoint load.
• New or modified files in user home directories, SSH key stores, or cron paths following a checkpoint load event.

Detection Strategies

• Hash and allow-list approved checkpoint artifacts; alert when a load targets a file outside the allow-list.
• Monitor Python interpreter processes for execution of shell utilities or network tools during model loading phases.
• Inspect checkpoint files for Python pickle opcodes referencing REDUCE, GLOBAL, or os.system-adjacent modules before use.

Monitoring Recommendations

• Enable process and command-line auditing on all ML training and inference hosts.
• Forward GPU host telemetry, EDR events, and network flows to a centralized data lake for correlation across clusters.
• Baseline normal child-process behavior of training jobs so deviations trigger investigation.

How to Mitigate CVE-2026-65110

Immediate Actions Required

• Stop loading Megatron-LM checkpoints from untrusted or unverified sources until the environment is patched.
• Apply the fixed Megatron-LM release referenced in the NVIDIA Security Repository.
• Rotate credentials, tokens, and SSH keys that were accessible from any host that recently loaded an unverified checkpoint.
• Run training workloads under dedicated, least-privilege service accounts isolated from production data stores.

Patch Information

NVIDIA has published the advisory and remediation guidance in its product security repository. Review the entry for bulletin 5881 at the NVIDIA Security Repository and upgrade to the fixed Megatron-LM version listed in the vendor bulletin. Validate the fix by re-running loader unit tests against known-good checkpoints after deployment.

Workarounds

• Load checkpoints only inside sandboxed containers with no network egress and read-only mounts for sensitive data.
• Convert incoming checkpoints to a safe tensor-only format, such as safetensors, before using them in training pipelines.
• Enforce cryptographic signing and verification of all internal checkpoints to block tampered artifacts.
• Restrict file system permissions so training users cannot write to directories used by shared services or orchestration agents.

# Configuration example: sandbox checkpoint loads with no network and read-only data
docker run --rm \
  --network=none \
  --read-only \
  --tmpfs /tmp:rw,size=512m \
  --user 10001:10001 \
  --cap-drop=ALL \
  -v /models/untrusted:/models:ro \
  megatron-lm:patched \
  python -m megatron.inspect_checkpoint --path /models/candidate.pt