CVE-2026-84366: Scrapy S3DownloadHandler Information Disclosure

CVE-2026-84366 Overview

CVE-2026-84366 affects Scrapy, a high-level web crawling and scraping framework for Python. Versions prior to 2.17.0 transmit signed Amazon S3 requests over plaintext HTTP by default. The S3DownloadHandler in scrapy/core/downloader/handlers/s3.py only uses HTTPS when request.meta["is_secure"] is explicitly set to a truthy value. Network attackers positioned between Scrapy and S3 can read AWS Authorization headers, X-Amz-Security-Token values, bucket paths, and object contents. Active attackers can also tamper with responses to poison scraped data, exports, and HTTP caches. The issue is classified as Cleartext Transmission of Sensitive Information [CWE-319].

Critical Impact
Passive attackers can harvest AWS credentials and session tokens from intercepted traffic, while active attackers can inject forged S3 responses that poison downstream crawl data and redirects.

Affected Products

Scrapy versions prior to 2.17.0
scrapy.core.downloader.handlers.s3.S3DownloadHandler component
Deployments issuing s3:// scheme requests with AWS credentials

Discovery Timeline

2026-09-01 - CVE-2026-84366 published to NVD
2026-09-02 - Last updated in NVD database

Technical Details for CVE-2026-84366

Vulnerability Analysis

Scrapy's S3DownloadHandler translates s3://bucket/key requests into virtual-hosted-style HTTP requests targeting {bucket}.s3.amazonaws.com. The handler selects the URL scheme based on request.meta["is_secure"], defaulting to http when the flag is unset. Scrapy then signs the request with AWS Signature and transmits the Authorization header, X-Amz-Security-Token (for temporary credentials), and payload in cleartext. Anyone observing the network path between Scrapy and Amazon S3 can capture these secrets and the returned object bytes. An active man-in-the-middle attacker can modify the response body, status code, and headers before Scrapy processes them.

Root Cause

The defect is a secure-by-default failure. The scheme selection logic used "https" if request.meta.get("is_secure") else "http", which resolves to plaintext whenever the caller omits the metadata key. Downstream code paths signed requests without validating the transport layer, so credential-bearing traffic flowed over TCP/80.

Attack Vector

Exploitation requires a network position between the Scrapy worker and s3.amazonaws.com. Suitable positions include compromised Wi-Fi, malicious upstream proxies, ISP-level interception, or a hostile transit provider. A passive attacker records the signed request to replay or extract the temporary session token. An active attacker rewrites S3 response bodies to inject attacker-controlled links, forge HTTP redirects, or poison a persistent HTTP cache, which then steers subsequent crawls toward attacker infrastructure.

python

# Patch from scrapy/core/downloader/handlers/s3.py
async def download_request(self, request: Request) -> Response:
    p = urlparse_cached(request)
-   scheme = "https" if request.meta.get("is_secure") else "http"
+   scheme = "http" if request.meta.get("is_secure") is False else "https"
    bucket = p.hostname
    path = p.path + "?" + p.query if p.query else p.path
    url = f"{scheme}://{bucket}.s3.amazonaws.com{path}"

Source: GitHub Scrapy Commit 9523e1e. The patch inverts the default so HTTPS is used unless callers explicitly opt out with request.meta["is_secure"] = False.

Detection Methods for CVE-2026-84366

Indicators of Compromise

Outbound TCP/80 connections from Scrapy hosts to *.s3.amazonaws.com or S3 regional endpoints.
HTTP requests containing Authorization: AWS4-HMAC-SHA256 or X-Amz-Security-Token headers observed on unencrypted flows.
Unexpected AWS CloudTrail GetObject events from IP ranges outside the Scrapy deployment.
Scraped datasets containing anomalous outbound links or content hashes not matching known S3 object versions.

Detection Strategies

Inspect network telemetry for cleartext HTTP requests to S3 endpoints originating from crawler hosts.
Audit Scrapy project configurations and code for s3:// URLs without a corresponding request.meta["is_secure"] = True.
Correlate AWS CloudTrail usage of specific access keys against the source IP allowlist for crawler infrastructure.

Monitoring Recommendations

Alert on any signed AWS request headers appearing in plaintext capture on egress mirrors or NDR sensors.
Track Scrapy version inventory across build pipelines and container images, flagging installs below 2.17.0.
Rotate and monitor AWS IAM credentials used by crawlers, watching for use from unexpected principals or regions.

How to Mitigate CVE-2026-84366

Immediate Actions Required

Upgrade Scrapy to version 2.17.0 or later across all crawler hosts, CI images, and container base layers.
Rotate AWS access keys and temporary session tokens that were used by vulnerable Scrapy deployments.
Review CloudTrail logs for anomalous s3:GetObject or s3:ListBucket activity tied to affected credentials.
Restrict Scrapy IAM roles to least privilege and scope them to specific buckets and prefixes.

Patch Information

The fix ships in Scrapy 2.17.0. See the GitHub Scrapy Release 2.17.0 notes and the GitHub Security Advisory GHSA-76g3-c3x4-crvx for details. The patched S3DownloadHandler now defaults to HTTPS and only uses HTTP when request.meta["is_secure"] is explicitly False.

Workarounds

For deployments that cannot upgrade immediately, set request.meta["is_secure"] = True on every S3-scheme request emitted by spiders and middlewares.
Force outbound TCP/80 to AWS S3 CIDR ranges to be blocked at the egress firewall to prevent cleartext fallback.
Route crawler traffic through an HTTPS-only forward proxy that rejects plaintext connections to S3 endpoints.

bash

# Upgrade Scrapy to the patched release
pip install --upgrade 'scrapy>=2.17.0'

# Verify installed version
python -c "import scrapy; print(scrapy.__version__)"

# Interim mitigation in a spider or middleware
# request.meta['is_secure'] = True