CVE-2026-42859: Neat VNC Stack Buffer Overflow Vulnerability

CVE-2026-42859 Overview

CVE-2026-42859 is a pre-authentication stack buffer overflow in Neat VNC, a Virtual Network Computing (VNC) server library used by Wayland-based remote desktop implementations. The flaw resides in the RSA-AES security type handler and affects all versions prior to 0.9.6. An unauthenticated remote attacker who can reach the VNC listening socket can trigger the overflow by sending a crafted handshake with an oversized client RSA public key. The condition is classified under [CWE-120] (Buffer Copy without Checking Size of Input). Exploitation results in at least a denial of service through a server crash, with potential for further memory corruption impact.

Critical Impact
Unauthenticated network attackers can crash any exposed Neat VNC server by sending a malformed RSA-AES handshake before authentication occurs.

Affected Products

Neat VNC versions prior to 0.9.6
Applications and distributions bundling the neatvnc library for VNC server functionality
Wayland compositors and remote desktop stacks depending on neatvnc

Discovery Timeline

2026-05-11 - CVE-2026-42859 published to NVD
2026-05-13 - Last updated in NVD database

Technical Details for CVE-2026-42859

Vulnerability Analysis

The vulnerability lives in rsa_aes_send_challenge within src/auth/rsa-aes.c. Neat VNC implements two RSA-based security types defined by the RFB protocol: security type 5 (RSA-AES) and security type 129 (RSA-AES-256). During the handshake, the server receives the client's RSA public key, then encrypts a server-generated challenge using that key. The encryption routine writes the resulting ciphertext into a fixed 1024-byte stack buffer without validating that the client's key size produces output that fits.

When an attacker advertises an oversized RSA public key, the encrypted challenge output exceeds the 1024-byte stack allocation. The write overflows adjacent stack memory, corrupting saved registers, return addresses, and stack canaries on platforms where they are enabled.

Root Cause

The root cause is missing input validation on the client-supplied RSA key length before performing RSA encryption into a stack buffer of fixed size. The code assumes the encrypted output will fit within 1024 bytes, but RSA output size scales linearly with key modulus length. A client controlling the key size therefore controls the destination write length.

Attack Vector

The attack vector is network-based and requires no authentication or user interaction. An attacker establishes a TCP connection to the VNC listening socket and initiates the RFB protocol handshake. The attacker selects security type 5 or 129, then sends a crafted client RSA public key with a modulus far larger than the server expects. The server invokes rsa_aes_send_challenge, which overflows the on-stack buffer during challenge encryption. The minimum confirmed impact is a server crash producing denial of service for all connected VNC clients.

The vulnerability is described in prose only as no verified public proof-of-concept code is published. Refer to the GitHub Security Advisory GHSA-567c-gpv8-qh9h and the upstream fix commit for technical details.

Detection Methods for CVE-2026-42859

Indicators of Compromise

Unexpected crashes or segmentation faults in processes linking libneatvnc, particularly during early connection handshake
Core dumps showing corrupted stack frames within rsa_aes_send_challenge or callers in src/auth/rsa-aes.c
Connection attempts from unknown sources negotiating RFB security type 5 or 129 followed by abrupt server termination

Detection Strategies

Monitor VNC server process restarts and abnormal exit codes correlated with inbound TCP sessions on the VNC listener
Inspect RFB handshake traffic for client RSA public key length fields that exceed expected sizes (typically 2048 or 4096-bit moduli)
Alert on repeated short-lived connections to VNC ports from a single source address that terminate before authentication completes

Monitoring Recommendations

Enable verbose logging in the host application embedding neatvnc to capture handshake failures and abort reasons
Forward VNC server logs and crash telemetry to a centralized SIEM for correlation across hosts
Track package versions of neatvnc across the fleet and flag any hosts still running versions below 0.9.6

How to Mitigate CVE-2026-42859

Immediate Actions Required

Upgrade neatvnc to version 0.9.6 or later on all systems running affected VNC server software
Restrict network exposure of VNC listening sockets to trusted management networks only, using firewall rules or host-based ACLs
Audit downstream applications and Linux distribution packages that embed neatvnc and apply vendor-rebuilt packages once available

Patch Information

The vulnerability is fixed in Neat VNC version 0.9.6. The upstream commit 1f6cd6b75cc167fed3a19a9d1552a1f662f6b337 adds bounds enforcement on the client RSA public key during the RSA-AES handshake, preventing oversized inputs from reaching the fixed-size stack buffer. Review the GitHub Security Advisory for full remediation guidance.

Workarounds

Disable RSA-AES security types (5 and 129) in the embedding application's configuration if upgrade is not immediately possible
Block inbound traffic to VNC ports at the perimeter and require VPN or SSH tunneling for remote desktop access
Run VNC services under a process supervisor with restart limits and resource isolation to contain repeated crash attempts

bash

# Configuration example: restrict VNC access via iptables to a management subnet
iptables -A INPUT -p tcp --dport 5900:5999 -s 10.0.10.0/24 -j ACCEPT
iptables -A INPUT -p tcp --dport 5900:5999 -j DROP