CVE-2026-28377: Grafana Tempo Key Exposure Vulnerability

CVE-2026-28377 Overview

A vulnerability in Grafana Tempo exposes the S3 SSE-C (Server-Side Encryption with Customer-Provided Keys) encryption key in plaintext through the /status/config endpoint. This information disclosure vulnerability potentially allows unauthorized users to obtain the encryption key used to protect trace data stored in Amazon S3 buckets.

The /status/config endpoint, designed to provide configuration status information, inadvertently includes the SSE-C encryption key material in its response. This exposure could allow attackers with network access to the Tempo instance to retrieve sensitive cryptographic keys, potentially compromising the confidentiality of all trace data encrypted with that key.

Critical Impact

Unauthorized exposure of S3 SSE-C encryption keys could allow attackers to decrypt sensitive trace data stored in S3, leading to significant data confidentiality breaches.

Affected Products

• Grafana Tempo (versions prior to security patch)

Discovery Timeline

• 2026-03-26 - CVE CVE-2026-28377 published to NVD
• 2026-03-26 - Last updated in NVD database

Technical Details for CVE-2026-28377

Vulnerability Analysis

This vulnerability represents a Sensitive Data Exposure issue where cryptographic key material is exposed through a configuration status endpoint. The root cause lies in insufficient sanitization of configuration data before it is returned to users through the /status/config API endpoint.

When Grafana Tempo is configured to use S3 SSE-C encryption for trace data storage, the encryption key is stored as part of the application configuration. The /status/config endpoint is designed to expose runtime configuration for debugging and monitoring purposes, but fails to redact or mask sensitive values such as encryption keys.

An attacker exploiting this vulnerability could obtain the SSE-C encryption key and subsequently use it to decrypt any trace data stored in the associated S3 bucket, provided they also have access to the encrypted objects. This represents a complete bypass of the data-at-rest encryption controls intended to protect trace information.

Root Cause

The vulnerability stems from improper handling of sensitive configuration values in the status endpoint implementation. The /status/config endpoint returns configuration data without properly redacting or masking sensitive cryptographic material. This is a common pattern in observability and monitoring tools where configuration endpoints are exposed for debugging purposes but inadvertently include secrets that should be protected.

Attack Vector

The attack is network-based and requires no authentication or user interaction. An attacker with network access to the Grafana Tempo instance can directly query the /status/config endpoint to retrieve the plaintext encryption key. The attack can be executed remotely with minimal complexity.

The exploitation process involves:

1. Identifying a Grafana Tempo instance with the vulnerable endpoint exposed
2. Sending an HTTP GET request to the /status/config endpoint
3. Parsing the response to extract the S3 SSE-C encryption key
4. Using the obtained key to decrypt trace data in the associated S3 bucket

Detection Methods for CVE-2026-28377

Indicators of Compromise

• Unusual access patterns to the /status/config endpoint, particularly from external or unauthorized IP addresses
• HTTP requests to Grafana Tempo status endpoints from unexpected network segments
• Evidence of S3 data access using SSE-C decryption from unfamiliar sources
• Audit log entries showing repeated queries to configuration endpoints

Detection Strategies

• Monitor HTTP access logs for requests to /status/config and similar configuration endpoints
• Implement network segmentation and access controls to restrict who can reach Tempo status endpoints
• Enable S3 access logging to detect unauthorized decryption attempts using the SSE-C key
• Deploy web application firewall rules to alert on access to sensitive configuration endpoints

Monitoring Recommendations

• Configure alerting for any external network access to Grafana Tempo management endpoints
• Implement rate limiting and anomaly detection for configuration endpoint access
• Regularly audit S3 bucket access logs for decryption operations with SSE-C keys
• Monitor for reconnaissance activity targeting observability infrastructure

How to Mitigate CVE-2026-28377

Immediate Actions Required

• Apply the security patch from Grafana as soon as it becomes available
• Restrict network access to the /status/config endpoint using firewall rules or network segmentation
• Rotate the S3 SSE-C encryption key immediately if exposure is suspected
• Review access logs to determine if the vulnerability may have been exploited

Patch Information

Grafana has released a security advisory addressing this vulnerability. Users should consult the Grafana Security Advisory CVE-2026-28377 for official patch information and upgrade instructions. Ensure your Grafana Tempo installation is updated to the patched version as recommended in the advisory.

Workarounds

• Implement network-level access controls to restrict access to Tempo status endpoints to trusted administrative networks only
• Use a reverse proxy or API gateway to filter requests to sensitive endpoints like /status/config
• Consider migrating to S3 SSE-KMS (AWS Key Management Service) instead of SSE-C, which provides better key isolation
• If using Kubernetes, implement NetworkPolicies to limit ingress to Tempo pods
• Rotate SSE-C encryption keys and re-encrypt existing data as a precautionary measure

# Example: Restrict access to Tempo status endpoint using iptables
# Allow only trusted admin network (10.0.0.0/24) to access Tempo on port 3200
iptables -A INPUT -p tcp --dport 3200 -s 10.0.0.0/24 -j ACCEPT
iptables -A INPUT -p tcp --dport 3200 -j DROP