CVE-2026-50538: LibVNCClient RCE Vulnerability

CVE-2026-50538 Overview

CVE-2026-50538 is a heap out-of-bounds write vulnerability in LibVNCClient, a library used to implement Virtual Network Computing (VNC) clients. The flaw affects versions 0.9.12 through 0.9.15. A malicious VNC server, or a man-in-the-middle attacker, can force a connecting libvncclient to write attacker-controlled data past the end of its framebuffer. The write occurs on the first FramebufferUpdate message after connection, requires no authentication, and triggers in default builds with default settings. Exploitation crashes the client and can be extended to overwrite application callback pointers, redirecting execution to attacker-chosen code.

Critical Impact
A single malicious FramebufferUpdate message can trigger remote code execution in any client linking libvncclient under its default configuration.

Affected Products

LibVNCClient 0.9.12
LibVNCClient 0.9.13, 0.9.14
LibVNCClient 0.9.15

Discovery Timeline

2026-08-21 - CVE-2026-50538 published to NVD
2026-08-24 - Last updated in NVD database

Technical Details for CVE-2026-50538

Vulnerability Analysis

The vulnerability resides in the Tight encoding decoder in src/libvncclient/tight.c. The Tight encoding compresses framebuffer updates using zlib streams. After decompression, the client passes decoded scan lines to a filter function that writes pixel data directly into client->frameBuffer. The decoder computed the number of decompressed rows from the zlib output buffer without validating that this number stayed within the rectangle's declared height. A server-controlled stream could yield more rows than the rectangle claimed to contain, causing filterFn() to write past the end of the framebuffer allocation. This condition is classified as CWE-122 (Heap-based Buffer Overflow).

Root Cause

The decoder performed a post-loop check comparing rowsProcessed against the rectangle height rh, but this validation ran after the out-of-bounds write already occurred. The attacker controls the length, contents, and offset of the write through the compressed stream, the pixel data, and the rectangle geometry declared in the FramebufferUpdate message.

Attack Vector

An attacker who operates a VNC server, or who intercepts a client's connection to a legitimate server, sends a crafted FramebufferUpdate message immediately after handshake completion. The malicious rectangle uses Tight encoding with a compressed payload that decodes to more scan lines than the rectangle height permits. The filterFn callback then writes attacker-supplied bytes into heap memory following the framebuffer, enabling denial of service or arbitrary code execution by overwriting adjacent function pointers.

      numRows = (bufferSize - zs->avail_out) / rowSize;

+      /* The decompressed stream is server-controlled and may yield more rows
+         than the rectangle's declared height.  filterFn() writes directly into
+         client->frameBuffer, so clamp here before writing to avoid running past
+         the framebuffer (heap out-of-bounds write).  The post-loop
+         "rowsProcessed != rh" check happens too late. */
+      if (numRows > rh - rowsProcessed) {
+	rfbClientLog("Tight: too many scan lines after decompression.\n");
+	return FALSE;
+      }
+
       filterFn(client, rx, ry+rowsProcessed, numRows);

       extraBytes = bufferSize - zs->avail_out - numRows * rowSize;

Source: LibVNC patch commit 540332be

Detection Methods for CVE-2026-50538

Indicators of Compromise

Client-side crashes or segmentation faults in processes linking libvncclient immediately after establishing a VNC session.
Log entries from rfbClientLog referencing decoding failures in the Tight decoder after applying the patch.
Outbound VNC connections (typically TCP 5900-5906) to untrusted or unexpected server addresses.

Detection Strategies

Inventory applications and packages linking libvncclient and correlate installed versions against the vulnerable range 0.9.12 through 0.9.15.
Monitor process behavior for VNC client executables spawning unexpected child processes or exhibiting heap corruption crash signatures.
Inspect network telemetry for VNC handshakes followed by unusually large or malformed FramebufferUpdate payloads using Tight encoding.

Monitoring Recommendations

Alert on VNC client processes writing to unusual memory regions or terminating with SIGSEGV shortly after network activity on VNC ports.
Track egress connections to VNC servers outside approved administrative networks.
Enable core dump collection on endpoints running VNC viewers to accelerate triage of exploitation attempts.

How to Mitigate CVE-2026-50538

Immediate Actions Required

Upgrade libvncserver/libvncclient to a release containing commit 540332be3e0acc566fa64da6f1b4680c72c724dd or later.
Rebuild and redistribute downstream applications that statically link libvncclient after the library update.
Restrict outbound VNC connections to trusted server addresses using host or network firewalls.

Patch Information

The issue is fixed in the LibVNC repository by commit 540332be3e0acc566fa64da6f1b4680c72c724dd. The patch adds a bounds check that rejects decompressed row counts exceeding the remaining rectangle height before filterFn() writes to the framebuffer. Details are documented in GitHub Security Advisory GHSA-v9pm-47h4-jcq8.

Workarounds

Avoid connecting VNC clients built on vulnerable libvncclient versions to untrusted servers.
Tunnel VNC traffic through authenticated, integrity-protected transports such as SSH to eliminate man-in-the-middle injection.
Disable or remove VNC client software on endpoints that do not require remote graphical administration.

bash

# Verify installed library version on Debian/Ubuntu
dpkg -l | grep -i libvncclient

# Update from source after fetching the patched commit
git clone https://github.com/LibVNC/libvncserver.git
cd libvncserver
git checkout 540332be3e0acc566fa64da6f1b4680c72c724dd
mkdir build && cd build
cmake .. && make && sudo make install