A new proof-of-concept reveals how attackers can turn Microsoft Copilot, the AI assistant embedded in Microsoft 365, into an unwitting accomplice for business email compromise (BEC) and large-scale wire fraud.
The demonstration shows that a single compromised employee account can escalate, with alarming speed, into full CEO account takeover and the theft of a quarter of a million dollars, with minimal technical effort from the attacker.
The attack begins the moment threat actors gain access to a regular employee’s inbox. Rather than relying on traditional “living off the land” techniques like PowerShell scripts or remote access tools, Barracuda’s researchers showed attackers instead abusing Copilot itself to accelerate every stage of the intrusion.
Their first move is establishing persistence: a simple Copilot prompt creates an inbox rule that silently redirects sign-in notifications to the Deleted Items folder, preventing the victim from noticing suspicious login alerts.
Hackers Weaponize Microsoft Copilot
With their foothold secured, attackers pivot to reconnaissance. Instead of manually sifting through months of email history, they ask Copilot to summarize organizational structure and surface active conversations, instantly identifying the company’s CEO as the next target.
Using context pulled from a real email thread between the victim and the CEO, attackers prompt Copilot to draft a convincing message written in the victim’s own tone and style, complete with a placeholder link disguised as an invoice confirmation.
When the CEO clicks the link, it routes through an adversary-in-the-middle proxy that intercepts the session token, allowing attackers to bypass multifactor authentication entirely and seize control of the CEO’s account. The same Copilot-generated inbox rule is reused to hide sign-in alerts and maintain stealth.
Once inside the CEO’s mailbox, the attackers ask Copilot for a “refresher on recent financial emails, including invoices, monetary values, and upcoming transfers.” Within seconds, Copilot surfaces a pending $247,500 wire transfer awaiting final approval, a discovery that would have taken a human attacker hours of manual searching.
Using the CEO’s authentic writing style, Copilot drafts an urgent email to the finance team requesting a bank account change for the transaction.
Because the message originates from the real CEO mailbox and passes every authentication check, it sails past traditional email security filters, and the finance team redirects the payment to the attacker’s account.
To keep the fraud hidden, attackers create a forwarding rule that silently reroutes the finance team’s replies to an external address, intercepting confirmation messages before the CEO ever sees them. Finally, Copilot is used again to locate and delete evidence of the entire scheme far faster than manual cleanup would allow.
Barracuda notes this technique isn’t unique to Copilot; any AI assistant with inbox access poses the same risk. The takeaway for security teams is clear: AI assistants act like knowledgeable insiders once an account is compromised, so monitoring AI-enabled accounts, inbox rule abuse, and anomalous session activity must become a core part of identity and email security strategy.
Strengthen Your SOC by Accelerating Threat Detection & Rapid Investigations. -> Integrate ANY.RUN With Your SOC Now.
