Safety researchers at Varonis have found a brand new assault that allowed them to exfiltrate person information from Microsoft Copilot utilizing a single malicious hyperlink.
Dubbed Reprompt, the assault bypassed the LLMs information leak protections and allowed for persistent session exfiltration even after the Copilot was closed, Varonis says.
The assault leverages a Parameter 2 Immediate (P2P) injection, a double-request method, and a chain-request method to allow steady, undetectable information exfiltration.
The Reprompt Copilot assault begins with the exploitation of the ‘q’ parameter, which is used on AI platforms to ship a person’s question or immediate by way of a URL. All it takes is for the person to click on on the hyperlink.
“By together with a particular query or instruction within the q parameter, builders and customers can mechanically populate the enter subject when the web page masses, inflicting the AI system to execute the immediate instantly,” Varonis explains.
A risk actor, the cybersecurity agency notes, may abuse the characteristic to make Copilot execute undesirable actions. The assault resulted in one-click compromise and, as a result of it leveraged the energetic person session, it endured after the chat was closed.Commercial. Scroll to proceed studying.
To forestall delicate info leaks, Copilot usually fetches URLs provided that a legitimate motive has been offered, and evaluations and alters delicate info earlier than returning it.
Nonetheless, Varonis found that the protections solely utilized to the preliminary request, and that they could possibly be bypassed by supplying every request a number of occasions.
The researchers added directions for Copilot to carry out every process twice, which resulted within the LLM leaking person info.
Particularly, they requested it to fetch a URL containing a secret phrase twice. Copilot eliminated the delicate info on the primary attempt, however included it within the second response.
Subsequent, the researchers developed a sequence request, the place Copilot retrieved the brand new instruction immediately from their assault server.
Every request instructed it each to exfiltrate extra person info and to fetch one other instruction, in a steady alternate with the server.
This ongoing alternate, Varonis notes, would permit an attacker to exfiltrate as a lot info as doable, requesting extra information based mostly on earlier responses.
Moreover, with all instructions despatched from the server, hidden within the follow-up requests, victims couldn’t decide what information was leaked after the preliminary immediate.
“Consumer-side monitoring instruments gained’t catch these malicious prompts, as a result of the actual information leaks occur dynamically throughout back-and-forth communication — not from something apparent within the immediate the person submits,” Varonis says.
Microsoft has resolved the underlying challenge. The assault doesn’t have an effect on enterprise clients utilizing Microsoft 365 Copilot, Varonis notes.
Associated: ‘EchoLeak’ AI Assault Enabled Theft of Delicate Knowledge by way of Microsoft 365 Copilot
Associated: Rethinking Safety for Agentic AI
Associated: Chrome Extensions With 900,000 Downloads Caught Stealing AI Chats
Associated: Militant Teams Are Experimenting With AI, and the Dangers Are Anticipated to Develop
