CVE-2026-77205: Libreswan FIPS Mode DOS Vulnerability

CVE-2026-77205 Overview

Libreswan running in FIPS mode contains a certificate cache handling flaw that allows attackers to lock out legitimate IPsec peers. The vulnerability stems from an incomplete fix for CVE-2026-14957, where the add_decoded_cert() function fails to call CERT_DestroyCertificate() on rejected malformed certificates. Rejected certificates remain in the Network Security Services (NSS) cache, causing any real certificate sharing the same serial number to be refused. An unauthenticated remote attacker can submit bogus certificates with chosen serial numbers to prevent legitimate peers from establishing connections until the service restarts.

Critical Impact
Remote attackers can deny IPsec connectivity to legitimate peers by poisoning the NSS certificate cache with malformed certificates sharing targeted serial numbers.

Affected Products

Libreswan IPsec implementation running in FIPS mode
Deployments relying on NSS for certificate validation
Systems that applied the earlier incomplete fix for CVE-2026-14957

Discovery Timeline

2026-10-13 - CVE CVE-2026-77205 published to NVD
2026-10-13 - Last updated in NVD database

Technical Details for CVE-2026-77205

Vulnerability Analysis

The flaw is a missing resource release [CWE-772] in Libreswan's certificate decoding path when operating under Federal Information Processing Standards (FIPS) mode. When a peer presents a malformed certificate, Libreswan rejects it but leaves the decoded certificate object resident in the NSS cache. Subsequent certificate lookups match the cached entry by serial number, causing valid certificates with the same serial to be treated as duplicates and rejected.

The network attack vector requires no privileges or user interaction. Exploitation complexity is high because the attacker must know or guess serial numbers used by legitimate peers. The impact is limited to availability of the IPsec service.

Root Cause

The add_decoded_cert() function omits a required call to CERT_DestroyCertificate() in the rejection path. The original fix for CVE-2026-14957 did not account for all failure branches, leaving decoded certificate handles leaked into the NSS cache. Because NSS indexes certificates by serial number, the stale entry collides with legitimate certificates presented later.

Attack Vector

An attacker initiates IKE negotiations against the Libreswan responder and presents malformed certificates whose serial numbers match those of legitimate peers. Each rejected certificate pollutes the NSS cache. Any subsequent connection attempt from the legitimate peer using a certificate with that serial number fails until the Libreswan service restarts and flushes the NSS cache.

No exploitation code is published. See the Libreswan CVE-2026-77205 Advisory for vendor technical details.

Detection Methods for CVE-2026-77205

Indicators of Compromise

Repeated IKE certificate validation failures in Libreswan logs referencing malformed or badly formed certificates
Legitimate peers unable to re-establish IPsec sessions despite valid credentials
Spike in inbound IKE_AUTH exchanges from unknown or spoofed sources presenting certificates with serial numbers matching known peers

Detection Strategies

Monitor Libreswan pluto daemon logs for certificate rejection messages correlated with subsequent authentication failures from legitimate peers
Alert when the same certificate serial number appears in both rejection and failure events within a short window
Track IKE session establishment rates per peer identity to detect peers that abruptly stop connecting

Monitoring Recommendations

Centralize Libreswan logs in a SIEM and build correlation rules on certificate validation events
Baseline normal IKE negotiation volume and alert on anomalous surges from untrusted sources
Review NSS database state after suspected incidents to confirm cache contents match issued certificates

How to Mitigate CVE-2026-77205

Immediate Actions Required

Apply the Libreswan patch that completes the CERT_DestroyCertificate() call in add_decoded_cert() as described in the vendor advisory
Restart the Libreswan pluto service to clear any poisoned NSS cache entries
Restrict IKE exposure to known peer IP ranges where network architecture allows

Patch Information

Refer to the Libreswan CVE-2026-77205 Details advisory for the fixed version and patch commit. Operators who applied the prior fix for the Libreswan CVE-2026-14957 Advisory must still install the CVE-2026-77205 update because the earlier fix was incomplete.

Workarounds

Limit inbound IKE/IPsec traffic to allow-listed peer addresses using firewall rules
Schedule periodic restarts of the Libreswan service to flush any accumulated cache poisoning until patching completes
Disable FIPS mode only if operationally and contractually acceptable, as the vulnerability is specific to the FIPS code path

bash

# Example firewall restriction limiting IKE to known peers
iptables -A INPUT -p udp --dport 500 -s <trusted_peer_ip> -j ACCEPT
iptables -A INPUT -p udp --dport 4500 -s <trusted_peer_ip> -j ACCEPT
iptables -A INPUT -p udp --dport 500 -j DROP
iptables -A INPUT -p udp --dport 4500 -j DROP

# Restart Libreswan to clear the NSS cache after patching
systemctl restart ipsec