CVE-2023-38408: OpenBSD OpenSSH RCE Vulnerability

CVE-2023-38408 Overview

CVE-2023-38408 is a remote code execution vulnerability in the PKCS#11 feature of ssh-agent in OpenSSH versions prior to 9.3p2. The flaw stems from an insufficiently trustworthy search path used when loading PKCS#11 provider libraries. When a user forwards their SSH agent to an attacker-controlled system, the attacker can load arbitrary shared libraries from /usr/lib into ssh-agent and chain them to achieve code execution on the victim's workstation. The issue exists because of an incomplete fix for CVE-2016-10009. Qualys researchers demonstrated working exploitation against default installations on several Linux distributions, making the issue widely applicable across enterprise environments.

Critical Impact

Attackers who control an SSH server can execute arbitrary code on any client that forwards an ssh-agent connection to that server, leading to full compromise of the user's workstation.

Affected Products

• OpenBSD OpenSSH versions before 9.3p2 (including 9.3 and 9.3p1)
• Fedora 37 and Fedora 38
• Downstream distributions including Debian, Gentoo, NetApp ONTAP, and Apple macOS (addressed in HT213940)

Discovery Timeline

• 2023-07-19 - Qualys Threat Research Unit publishes analysis and OpenBSD commits the upstream fix
• 2023-07-20 - CVE-2023-38408 published to NVD
• 2024-11-21 - Last updated in NVD database

Technical Details for CVE-2023-38408

Vulnerability Analysis

The vulnerability is classified as an Untrusted Search Path issue [CWE-428] within the PKCS#11 provider loading logic of ssh-agent. When a client forwards its agent to a remote host, the remote host can send SSH_AGENTC_ADD_SMARTCARD_KEY messages back to the local agent. These messages instruct the agent to dlopen() a shared library by path. The pre-patch code only restricted absolute paths but allowed any library reachable via the dynamic linker's search path, including /usr/lib. Qualys demonstrated that by repeatedly loading and unloading specific stock libraries from /usr/lib, an attacker could side-effect the agent's memory layout and chain gadgets to reach system() or equivalent execution primitives. Each forwarded message executes inside the trusted local ssh-agent process, bypassing the protections normally afforded by agent forwarding.

Root Cause

The root cause is the agent's willingness to load arbitrary shared objects requested by a forwarded peer without verifying that the object is a legitimate PKCS#11 provider. The original fix for CVE-2016-10009 blocked path traversal but did not restrict which libraries within standard system directories could be loaded. Standard system libraries are not designed to be safely loaded and unloaded into an arbitrary host process, and their constructors and destructors can be weaponized.

Attack Vector

Exploitation requires that a victim connect to an attacker-controlled SSH server with agent forwarding enabled (ssh -A or ForwardAgent yes). Once connected, the attacker uses the forwarded agent socket to issue add-smartcard-key requests that load chosen libraries from /usr/lib. By chaining the side effects of multiple library load and unload operations, the attacker achieves arbitrary code execution in the context of the user running ssh-agent on the originating host.

/* Patch: usr.bin/ssh/ssh-pkcs11.c
 * Terminate process if requested to load a PKCS#11 provider
 * that does not contain the expected symbols.
 */
-/* $OpenBSD: ssh-pkcs11.c,v 1.56 2023/03/08 05:33:53 tb Exp $ */
+/* $OpenBSD: ssh-pkcs11.c,v 1.57 2023/07/19 13:55:53 djm Exp $ */
 /*
  * Copyright (c) 2010 Markus Friedl.  All rights reserved.
  * Copyright (c) 2014 Pedro Martelletto. All rights reserved.

Source: OpenBSD Commit f03a4fa

/* Patch: usr.bin/ssh/misc.c
 * Ensure FIDO/PKCS11 libraries contain expected symbols before loading.
 */
-/* $OpenBSD: misc.c,v 1.183 2023/07/14 07:44:21 dtucker Exp $ */
+/* $OpenBSD: misc.c,v 1.184 2023/07/19 14:02:27 djm Exp $ */

Source: OpenBSD Commit f8f5a6b

Detection Methods for CVE-2023-38408

Indicators of Compromise

• Unexpected dlopen() calls from the ssh-agent process targeting libraries under /usr/lib that are not legitimate PKCS#11 providers.
• Child processes spawned by ssh-agent, which should not occur during normal operation.
• Outbound network connections originating from ssh-agent or processes that inherit its environment.
• SSH client sessions using ForwardAgent yes to untrusted or recently registered hosts.

Detection Strategies

• Hunt for process lineage where ssh-agent is the parent of unexpected binaries such as shells, interpreters, or network utilities.
• Inspect auditd or eBPF telemetry for execve and open events initiated by ssh-agent referencing shared objects outside the OpenSSH package.
• Correlate SSH client logs showing agent forwarding with subsequent anomalous activity on the originating workstation.

Monitoring Recommendations

• Track installed OpenSSH versions across the fleet and alert on hosts running versions earlier than 9.3p2.
• Monitor SSH client configuration files for ForwardAgent yes directives applied to wildcard host patterns.
• Log and review every interactive SSH session that requests agent forwarding to externally hosted systems.

How to Mitigate CVE-2023-38408

Immediate Actions Required

• Upgrade OpenSSH to version 9.3p2 or later on all clients that may forward an agent connection.
• Apply vendor-supplied patches from Debian, Fedora, Gentoo, NetApp, and Apple (macOS update HT213940).
• Disable agent forwarding by default and re-enable it only for explicitly trusted destinations.
• Audit user ~/.ssh/config files and centrally managed SSH configurations for unsafe ForwardAgent defaults.

Patch Information

OpenSSH 9.3p2 introduces three upstream fixes: commit 7bc29a9 disallows remote addition of FIDO and PKCS#11 provider libraries to ssh-agent by default, commit f03a4fa terminates the process if a requested provider lacks expected symbols, and commit f8f5a6b validates that FIDO and PKCS#11 libraries contain the expected entry points before loading. See the OpenSSH 9.3p2 release notes and the Qualys advisory for full details.

Workarounds

• Set ForwardAgent no globally in /etc/ssh/ssh_config and only enable it per-host for trusted endpoints.
• Use the -o AddKeysToAgent=confirm option or ssh-agent -P allowlist (available in newer OpenSSH releases) to restrict which providers may be loaded.
• Prefer ProxyJump (ssh -J) over agent forwarding for multi-hop SSH workflows.
• Use hardware-backed keys with FIDO authenticators (ed25519-sk) that do not require forwarding the full agent.

# /etc/ssh/ssh_config - safe defaults
Host *
    ForwardAgent no
    AddKeysToAgent confirm

# Restrict allowed PKCS#11 providers in ssh-agent (OpenSSH >= 9.3p2)
ssh-agent -P '/usr/lib64/pkcs11/opensc-pkcs11.so'