CVE-2026-5740: Mattermost Server DOS Vulnerability

CVE-2026-5740 Overview

CVE-2026-5740 is a denial-of-service vulnerability in Mattermost Server affecting versions 11.6.x through 11.6.0, 11.5.x through 11.5.3, 11.4.x through 11.4.4, and 10.11.x through 10.11.14. The flaw resides in the handling of msgpack-encoded WebSocket frames on the public WebSocket endpoint. Mattermost fails to validate frame metadata before allocating memory, allowing an unauthenticated remote attacker to crash the server process with a single crafted binary WebSocket message. The advisory is tracked as Mattermost Advisory ID MMSA-2026-00647 and is classified under CWE-789 (Memory Allocation with Excessive Size Value).

Critical Impact

An unauthenticated attacker can trigger a full service outage for all Mattermost users by sending a single crafted binary WebSocket message to the public endpoint.

Affected Products

• Mattermost Server 11.6.x through 11.6.0
• Mattermost Server 11.5.x through 11.5.3, and 11.4.x through 11.4.4
• Mattermost Server 10.11.x through 10.11.14

Discovery Timeline

• 2026-05-22 - CVE-2026-5740 published to NVD
• 2026-05-22 - Last updated in NVD database

Technical Details for CVE-2026-5740

Vulnerability Analysis

The vulnerability is a resource exhaustion flaw triggered through the public WebSocket endpoint of Mattermost Server. Mattermost accepts binary WebSocket frames encoded with MessagePack (msgpack), a compact binary serialization format. The server reads size metadata embedded in the msgpack payload and allocates memory based on the declared size before validating that the value is reasonable or that subsequent bytes match the declared length.

An attacker crafts a binary WebSocket frame whose msgpack header declares an excessive size value. The Go runtime attempts to allocate the requested buffer, exhausting available memory and crashing the server process. Because the WebSocket endpoint is reachable without authentication, the attack does not require valid credentials, team membership, or any prior interaction.

The resulting process crash terminates all active WebSocket sessions and HTTP request handling, producing a full service outage until the process restarts. Repeated requests can keep the service in a crash loop.

Root Cause

The root cause is missing size validation between msgpack frame parsing and memory allocation, classified as CWE-789. The decoder trusts the declared length field in the msgpack header without bounding it against the actual frame size or a configured maximum, violating safe deserialization practice.

Attack Vector

Exploitation requires only network reachability to the Mattermost WebSocket endpoint. The attacker opens a WebSocket connection to the public endpoint, transmits a single binary frame containing the malformed msgpack payload, and the server process terminates. No authentication, user interaction, or specific privileges are required. See the Mattermost Security Updates advisory for additional context.

Detection Methods for CVE-2026-5740

Indicators of Compromise

• Unexpected Mattermost server process crashes or restarts correlated with active WebSocket connections.
• Out-of-memory (OOM) kill events in system logs targeting the mattermost process.
• Binary WebSocket frames sent to /api/v4/websocket from unauthenticated or low-reputation source IPs.
• Spikes in resident memory consumption immediately preceding process termination.

Detection Strategies

• Monitor application logs for abnormal termination signals and panic traces in the Mattermost process.
• Inspect reverse proxy and load balancer logs for WebSocket upgrade requests followed by immediate disconnects.
• Alert on repeated WebSocket connection attempts from the same source IP that precede service restarts.

Monitoring Recommendations

• Track process uptime and memory utilization for Mattermost worker nodes with a baseline alert on rapid growth.
• Enable WebSocket frame size logging at the reverse proxy layer (NGINX, HAProxy) to record unusually large frames.
• Correlate crash events with network telemetry to identify the source IP issuing the malformed frames.

How to Mitigate CVE-2026-5740

Immediate Actions Required

• Upgrade Mattermost Server to a fixed release listed in the Mattermost Security Updates advisory.
• Restrict access to the WebSocket endpoint behind a VPN or IP allowlist where business requirements permit.
• Configure a reverse proxy to enforce maximum WebSocket frame and message size limits.

Patch Information

Mattermost has released fixed versions addressed in advisory MMSA-2026-00647. Administrators running 11.6.x, 11.5.x, 11.4.x, or 10.11.x branches must update to the patched build for their respective branch. Refer to the Mattermost Security Updates page for the exact fixed version numbers and upgrade instructions.

Workarounds

• Place Mattermost behind a reverse proxy that caps WebSocket frame size below the threshold required to trigger the allocation, such as the NGINX client_max_body_size and equivalent WebSocket buffer limits.
• Deploy rate limiting on WebSocket upgrade requests at the edge to slow repeated exploitation attempts.
• Run Mattermost under a process supervisor with memory cgroup limits so the OS terminates the process before the host is affected, and enable automatic restart.

# Example NGINX configuration limiting WebSocket payload size
http {
    client_max_body_size 1m;
    proxy_read_timeout 60s;

    server {
        location /api/v4/websocket {
            proxy_pass http://mattermost_backend;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_buffer_size 8k;
            proxy_buffers 4 8k;
        }
    }
}