In a recent disclosure, cybersecurity experts revealed a Spectre attack targeting Cloudflare Workers, which managed to extract a JSON Web Token (JWT) from a co-located Worker in the production environment at a speed of up to 12 bits per second. This new attack rate significantly surpasses a previous demonstration from 2021, which achieved only 2 bits per minute.
The research involved setting up both attacker and victim Workers, with the JWT deliberately placed in the victim’s memory. Importantly, the researchers emphasized that no customer data was accessed during their experiments. Cloudflare has since mitigated the threat by enhancing its Dynamic Process Isolation (DyPrIs), integrating the V8 Sandbox, and implementing Memory Protection Keys (MPKs) for in-process isolation, ensuring no signs of active exploitation have been observed over the past three years.
Details of the Spectre Attack
The attack exploited the way Cloudflare Workers operate, running code from multiple users in separate V8 isolates within a single operating-system process. This approach relies on language-level isolation rather than strict process isolation to minimize startup latency. However, as Cloudflare acknowledged, this setup can lead to potential cross-tenant data leakage if a memory read occurs within a shared Worker process.
The attack conditions required the malicious and victim Workers to be located in separate V8 isolates within the same Worker process. The attacker executed valid code within its isolate, without exploiting V8 software vulnerabilities or escaping the sandbox environment. Cloudflare has addressed these vulnerabilities by freezing or coarsening local timing sources during CPU execution, preventing shared memory or multithreading exposure to Worker scripts.
Challenges in Detection and Isolation
Research findings highlighted that WebSocket communications could serve as a remote timing source, while Durable Objects could sustain a single Worker isolate for extended periods, ranging from five to over twenty hours. The researchers observed that DyPrIs, designed to isolate suspicious scripts into a separate process post-invocation, failed to isolate long-lived Durable Object invocations promptly.
Moreover, the researchers discovered that extensive WebSocket input/output activity increased instruction translation lookaside buffer (iTLB) activity, reducing the effectiveness of DyPrIs’ normalized branch-misprediction signal, which is used for threat detection. Cloudflare recognized these issues as intrinsic limitations of the detection methodology rather than mere implementation oversights.
Technical and Defensive Measures
The research was conducted on Linux servers equipped with AMD EPYC Zen 2 and Zen 3 processors. Tests were performed during low CPU utilization periods to optimize results, although higher system loads did slow down the leakage rate. Even under heavy load, the attack remained feasible.
This recent disclosure follows nearly five years after Cloudflare and TU Graz initially demonstrated a remote Spectre attack against Workers. The earlier research reported a 0.61% false-positive rate, asserting that DyPrIs offered comparable security assurances to strict process isolation against evaluated Spectre attacks. To counteract these vulnerabilities, Cloudflare has rolled out several hardening measures, such as improving DyPrIs detection capabilities, limiting transient access with the V8 Sandbox, and using MPK-based in-process isolation to enhance security.
Cloudflare’s latest updates include a random MPK assignment strategy, capturing approximately 92% of cross-isolate accesses, alongside a stringent rotating layout to close any remaining security gaps. These enhancements reflect ongoing efforts to bolster the platform’s defenses against emerging threats.
