CVE-2026-23157: Linux Kernel Btrfs DoS Vulnerability

CVE-2026-23157 Overview

A deadlock vulnerability has been identified in the Linux kernel's Btrfs filesystem implementation. The issue occurs in the btree_writepages() function where an internal 32MiB dirty metadata threshold can conflict with cgroup memory limits, causing over 1000 processes to become stuck waiting at io_schedule_timeout() in balance_dirty_pages(). This results in system hangs and potential kernel coredumps.

Critical Impact

Systems running affected Linux kernel versions with Btrfs filesystems and cgroups with constrained memory limits may experience complete system hangs due to a deadlock between Btrfs dirty page thresholds and cgroup memory constraints.

Affected Products

• Linux kernel versions before v6.18
• Linux kernel v6.4-based distributions
• All current LTS kernels without the AS_KERNEL_FILE enhancement

Discovery Timeline

• 2026-02-14 - CVE CVE-2026-23157 published to NVD
• 2026-02-18 - Last updated in NVD database

Technical Details for CVE-2026-23157

Vulnerability Analysis

This vulnerability represents a race condition and deadlock scenario within the Linux kernel's Btrfs filesystem dirty page management. The root cause involves a conflict between Btrfs's internal 32MiB dirty metadata threshold and cgroup memory constraints.

When a cgroup has a small memory allocation (such as 16MB dirty limit), tasks within that cgroup can dirty pages in the Btrfs btree inode. The balance_dirty_pages() function prevents tasks from exiting until dirty pages drop below the cgroup limit. However, Btrfs's btree_writepages() function has an internal threshold that prevents writeback until at least 32MiB of dirty metadata accumulates.

This creates a circular dependency: Btrfs won't write back pages until more are dirtied, but the cgroup won't allow more pages to be dirtied until existing ones are written back. The result is a deadlock where all processes touching the btree inode are put to sleep indefinitely.

Root Cause

The deadlock stems from a design conflict between two independent subsystems:

1. Btrfs Metadata Batching: The btree_writepages() function implements a 32MiB threshold to batch metadata writes, preventing premature writeback of tree blocks that might need to be re-COWed for subsequent modifications.

2. Cgroup Dirty Page Throttling: The memory cgroup subsystem enforces dirty page limits to prevent any single cgroup from consuming excessive dirty memory. When a task exceeds its cgroup's dirty limit, it blocks in balance_dirty_pages() until dirty pages are reduced.

When the Btrfs threshold (32MiB) exceeds the cgroup's dirty limit (e.g., 16MB), neither subsystem can make progress, resulting in a deadlock condition.

Attack Vector

This vulnerability manifests through resource exhaustion under specific system configurations. The attack vector involves systems running Btrfs filesystems with memory cgroups that have restricted dirty page limits. Any workload that generates sufficient btree inode dirty pages (around 28MiB in the reported case) while staying under Btrfs's 32MiB threshold can trigger the deadlock.

The vulnerability does not require malicious intent - normal system operations under constrained cgroup configurations can trigger the condition. However, an attacker with local access could potentially craft workloads to deliberately trigger the deadlock and cause denial of service.

Detection Methods for CVE-2026-23157

Indicators of Compromise

• Large numbers of processes (potentially 1000+) stuck in D (uninterruptible sleep) state
• Processes waiting at io_schedule_timeout() in balance_dirty_pages() visible in kernel stack traces
• System unresponsiveness affecting Btrfs-mounted filesystems
• Kernel coredumps triggered by watchdog timeouts

Detection Strategies

• Monitor for abnormally high counts of processes in uninterruptible sleep state using ps aux | grep ' D '
• Check /proc/[pid]/stack for processes stuck in balance_dirty_pages call paths
• Review /proc/meminfo and cgroup memory statistics for dirty page accumulation
• Implement kernel watchdog monitoring for detection of hung task warnings

Monitoring Recommendations

• Configure kernel hung task detection with appropriate timeouts (kernel.hung_task_timeout_secs)
• Monitor cgroup memory statistics via /sys/fs/cgroup/memory/*/memory.stat for dirty page thresholds
• Set up alerts for Btrfs-specific metrics including btree inode dirty bytes
• Implement process state monitoring to detect mass process blocking conditions

How to Mitigate CVE-2026-23157

Immediate Actions Required

• Apply the kernel patches from the referenced git commits to affected systems
• Review cgroup memory configurations, particularly dirty page limits relative to Btrfs thresholds
• Consider temporarily increasing cgroup dirty page limits above 32MiB for critical systems
• Plan kernel updates to versions containing the fix

Patch Information

The vulnerability has been addressed in the Linux kernel stable tree. The fix modifies btree_writepages() to respect external caller requests for writeback regardless of the internal 32MiB threshold, while maintaining the threshold behavior for internal Btrfs callers using btrfs_btree_balance_dirty().

Patches are available at:

• Kernel Git Commit 4e15915
• Kernel Git Commit 629666d

For newer kernels (v6.18+), commit b55102826d7d ("btrfs: set AS_KERNEL_FILE on the btree_inode") provides an alternative mitigation by charging btree inode pages to the root cgroup.

Workarounds

• Increase cgroup dirty page limits to values exceeding 32MiB for workloads using Btrfs
• Avoid using memory-constrained cgroups with Btrfs filesystems on unpatched kernels
• Consider alternative filesystems for containerized or cgroup-constrained workloads until patches are applied
• Monitor and restart affected processes if deadlock conditions are detected

# Configuration example - Increase cgroup dirty ratio
# For cgroup v1:
echo 67108864 > /sys/fs/cgroup/memory/<cgroup_name>/memory.dirty_limit_in_bytes

# For cgroup v2 (check dirty ratio in memory controller):
# Ensure dirty limits exceed 32MiB (33554432 bytes minimum recommended)
cat /sys/fs/cgroup/<cgroup_path>/memory.stat | grep dirty