Introduction
Recent insights reveal that cybercriminals can transform Microsoft Copilot, an AI assistant in Microsoft 365, into a tool for business email compromise (BEC) and financial fraud. This method allows attackers to commandeer CEO accounts and redirect substantial wire transfers with minimal effort.
The proof-of-concept illustrates the rapid escalation from a compromised employee account to controlling a CEO’s account, resulting in a $250,000 theft. This tactic requires little technical prowess from the attackers.
The Initial Breach
The process starts when attackers gain access to an employee’s email account. Instead of relying on conventional hacking methods like PowerShell or remote access, researchers from Barracuda demonstrated how attackers utilize Copilot to enhance every stage of their intrusion.
The initial step involves establishing persistence. By using a simple Copilot command, attackers can create an inbox rule that redirects sign-in notifications to the Deleted Items folder, effectively concealing any suspicious login attempts from the victim.
Reconnaissance and Targeting
After securing their position, the attackers conduct reconnaissance. Rather than manually reviewing extensive email archives, they instruct Copilot to summarize the organization’s structure and highlight active conversations, quickly identifying the CEO as the next target.
Leveraging context from existing email threads, attackers use Copilot to craft a convincing message in the victim’s style, including a disguised link masquerading as an invoice confirmation.
Executing the Attack
Once the CEO interacts with the link, it is routed through a proxy that intercepts the session token, allowing attackers to bypass multifactor authentication and seize control of the CEO’s account. The same Copilot-generated rule is reused to hide notifications and maintain stealth.
With access to the CEO’s emails, attackers request Copilot to scan for recent financial communications, quickly identifying a pending $247,500 transfer. Copilot drafts a seemingly authentic email to the finance team to change the transaction’s bank account details.
The email, sent from the CEO’s account, passes all security checks, leading the finance team to redirect the funds to the attacker. To remain undetected, attackers create forwarding rules to intercept any replies and use Copilot to erase traces of their activities.
Conclusion and Security Implications
Barracuda emphasizes that this vulnerability is not exclusive to Copilot; any AI assistant with email access poses similar risks. The critical lesson for security teams is that AI assistants can function like knowledgeable insiders post-compromise. Therefore, monitoring AI-enabled accounts, inbox rules, and unusual session activities must be integral to email and identity security strategies.
Enhancing security operations by accelerating threat detection and rapid investigations is crucial. Integrating tools like ANY.RUN can help strengthen your Security Operations Center (SOC) against such advanced threats.
