In a concerning new trend, cybercriminals are embedding hidden commands within everyday emails, documents, and online ads, aiming to exploit artificial intelligence (AI) systems designed to protect digital communications. These tactics, often invisible to human eyes, directly target AI mail agents to potentially manipulate their actions or expose sensitive data.
Hidden Commands Target AI Systems
Referred to as indirect prompt injection, this method turns AI assistants into unintended targets by embedding commands that appear innocuous to people but are interpreted as legitimate requests by AI systems. According to Proofpoint, discussions and sales of such techniques are growing on underground forums, indicating the development of tools to generate these hidden prompts, although widespread misuse has not yet been observed.
This shift is significant as AI increasingly oversees incoming emails, attachments, calendars, and web content. Cybercriminals seek to exploit this automation, adding complexity to traditional phishing and social engineering attacks. Proofpoint’s report, shared with Cyber Security News, highlights that while these tools are currently experimental, organizations should prepare for their potential use.
Mechanisms of Indirect Prompt Injection
Indirect prompt injection occurs when AI systems mistakenly interpret embedded instructions as legitimate commands during content processing. Unlike direct prompt attacks, where commands are entered into chatbots, this approach hides commands within content that AI agents automatically read. Examples include emails with invisible ‘white-on-white’ text, posing as standard communications while influencing AI reading agents.
Similarly, attachments, like PDFs or DOCX files appearing as ordinary documents, may contain concealed text that affects AI scanning tools. The threat level varies with the AI system’s permissions, becoming more severe if the AI can act independently without human approval.
Security experts advocate treating external content as untrusted, even in familiar formats, and suggest separating untrusted text from system instructions, limiting AI access, and requiring human verification for critical actions to mitigate risks.
Expanding Risk with Calendar Invites
Criminals are also crafting prompt-injection tools for calendar invitations, embedding malicious commands in event descriptions disguised as meeting agendas. Unlike past phishing tactics, AI agents may automatically process these invitations as part of routine work, echoing the risks of weaponized calendar files.
Proofpoint’s findings also reveal interest in embedding prompts in malicious ads and webpages, using elements like HTML or image alternative text that humans might overlook but AI systems would process.
Organizations are encouraged to assess AI tool integration with emails, files, calendars, and web content, applying access controls to protect sensitive data and actions. Employees should continue flagging suspicious communications, while security teams monitor evolving phishing strategies that combine traditional delivery methods with AI-targeted manipulation.
Though no large-scale campaigns using these techniques have been noted yet, the active development and marketing of related tools suggest attackers are preparing for a future where AI systems are integral to the attack landscape.
