CVE-2022-23471: Containerd CRI Memory Leak DoS Vulnerability

CVE-2022-23471 Overview

CVE-2022-23471 is a Memory Leak vulnerability discovered in containerd, an open source container runtime widely used in Kubernetes environments. A bug was found in containerd's CRI (Container Runtime Interface) implementation where an authenticated user can exhaust memory on the host system through improper goroutine handling.

In the CRI stream server, a goroutine is launched to handle terminal resize events when a TTY is requested. If the user's process fails to launch due to, for example, a faulty command, the goroutine becomes stuck waiting to send without a receiver, resulting in a memory leak that can lead to denial of service conditions.

Critical Impact

Authenticated attackers can exhaust host memory through repeated failed container executions with TTY requests, potentially causing denial of service across the entire Kubernetes cluster or container infrastructure.

Affected Products

• containerd versions prior to 1.6.12
• containerd versions prior to 1.5.16
• Kubernetes and crictl deployments configured to use containerd's CRI implementation

Discovery Timeline

• December 7, 2022 - CVE-2022-23471 published to NVD
• November 21, 2024 - Last updated in NVD database

Technical Details for CVE-2022-23471

Vulnerability Analysis

This vulnerability affects the CRI stream server component of containerd, which is responsible for handling container I/O operations. The flaw exists in the terminal resize event handling mechanism within the remote command execution workflow.

When a container is executed with a TTY (terminal) request, containerd spawns a dedicated goroutine to manage terminal resize events. Under normal operation, this goroutine communicates with the container process through channels. However, when the container process fails to launch—such as when an invalid or faulty command is specified—the goroutine remains blocked in a send operation with no corresponding receiver to consume the message.

This creates a resource leak where each failed execution attempt with a TTY request leaves behind an orphaned goroutine consuming memory. An attacker with permissions to execute commands in containers can repeatedly trigger this condition, gradually exhausting available memory on the host system.

Root Cause

The root cause is improper resource management in the pkg/cri/streaming/remotecommand/httpstream.go file. The goroutine responsible for handling terminal resize events lacks proper context cancellation or timeout mechanisms that would allow it to terminate gracefully when the target process fails to start. This results in CWE-400 (Uncontrolled Resource Consumption) and CWE-401 (Missing Release of Memory after Effective Lifetime).

Attack Vector

The attack vector requires network access and low-privilege authentication. An attacker must have permissions to execute commands in running containers, which is a common capability in multi-tenant Kubernetes environments. The attack can be executed remotely through the Kubernetes API or crictl commands:

1. Attacker authenticates to the container orchestration platform
2. Attacker initiates container exec commands with TTY enabled and intentionally faulty commands
3. Each failed execution leaks memory through orphaned goroutines
4. Repeated attacks exhaust host memory, causing denial of service

The security patch introduces proper context handling to ensure goroutines are properly terminated when container processes fail:

 package remotecommand
 
 import (
+	gocontext "context"
 	"encoding/json"
 	"errors"
 	"fmt"

Source: GitHub Commit

The fix adds Go context support to enable proper cancellation of goroutines when the parent process fails, preventing memory leaks.

Detection Methods for CVE-2022-23471

Indicators of Compromise

• Unusual memory consumption growth on container host systems running containerd
• Increasing number of goroutines in containerd process over time
• Failed container exec operations with TTY requests in audit logs
• Repeated exec API calls from the same user or service account with command failures

Detection Strategies

• Monitor containerd process memory usage for abnormal growth patterns using tools like Prometheus with cAdvisor
• Implement audit logging for container exec operations and analyze for patterns of failed TTY executions
• Deploy SentinelOne Singularity for Cloud to detect anomalous container runtime behavior and resource exhaustion attacks
• Create alerts for high rates of failed container exec operations within short time windows

Monitoring Recommendations

• Set up memory usage thresholds for containerd daemon processes with alerting when exceeded
• Enable Kubernetes audit logging to track exec operations and correlate with containerd metrics
• Implement rate limiting on container exec operations at the API server level
• Use SentinelOne's Kubernetes protection to identify unauthorized or suspicious exec activity patterns

How to Mitigate CVE-2022-23471

Immediate Actions Required

• Upgrade containerd to version 1.6.12 or later for the 1.6.x branch
• Upgrade containerd to version 1.5.16 or later for the 1.5.x branch
• Review and restrict permissions for users and service accounts that can execute commands in containers
• Monitor container hosts for signs of memory exhaustion attacks

Patch Information

The vulnerability has been fixed in containerd versions 1.6.12 and 1.5.16. The fix was implemented through commit a05d175400b1145e5e6a735a6710579d181e7fb0, which introduces proper context handling for goroutine lifecycle management in the CRI stream server.

The release notes for version 1.6.12 specifically address this issue:

+# commit to be tagged for new release
+commit = "HEAD"
+
+project_name = "containerd"
+github_repo = "containerd/containerd"
+match_deps = "^github.com/(containerd/[a-zA-Z0-9-]+)$"
+
+# previous release
+previous = "v1.6.11"
+
+pre_release = false
+
+preface = """\
+The twelfth patch release for containerd 1.6 contains a fix for CVE-2022-23471.
+
+### Notable Updates
+* **Fix goroutine leak during Exec in CRI plugin** ([GHSA-2qjp-425j-52j9](https://github.com/containerd/containerd/security/advisories/GHSA-2qjp-425j-52j9))
+
+See the changelog for complete list of changes"""

Source: GitHub Commit

For additional details, refer to the GitHub Security Advisory and Gentoo GLSA 202401-31.

Workarounds

• Ensure that only trusted container images and commands are used in your environment
• Restrict container exec permissions to only trusted users through RBAC policies
• Implement network policies to limit which services can access the container runtime API
• Deploy admission controllers to validate and restrict exec operations on containers

# Example: Kubernetes RBAC to restrict exec permissions
# Create a ClusterRole that denies exec access
kubectl create clusterrole deny-exec --verb=create --resource=pods/exec --dry-run=client -o yaml | \
  sed 's/verbs:/verbs: []/; s/- create//' | kubectl apply -f -

# Verify containerd version
containerd --version

# Check for available updates on Debian/Ubuntu
apt-cache policy containerd.io