Microsoft has raised alarms over a recent phishing campaign that cleverly used AI technology to conceal its malicious code and slip past email security filters. According to the company, the attackers appeared to have harnessed a large language model (LLM) to generate complex, machine-crafted code that mimicked legitimate business content. 

AI-Generated Code Used to Hide Malicious Payload

Microsoft's security researchers discovered that the phishing emails carried SVG files (scalable vector graphics) that weren't what they seemed. Hidden inside these files was obfuscated code—a tangled, verbose mess that looked synthetically generated rather than hand-written.

"Appearing to be aided by a large language model (LLM), the activity obfuscated its behavior within an SVG file, leveraging business terminology and a synthetic structure to disguise its malicious intent," Microsoft wrote.

When analyzed using Microsoft Security Copilot, the company's AI-driven security tool, the code was flagged as something "not typically written by a human from scratch," given its complexity, redundancy, and lack of practical purpose. In short, the attackers had used AI to make their malicious code look convincingly technical and harmless—enough to trick both humans and some filters.

How the Attack Worked

The phishing emails came from a compromised small business email account, making them appear trustworthy. They were disguised as file-sharing notifications, prompting recipients to open an attached SVG file.

Once opened, the victim was redirected to a fake login page designed to harvest credentials—an old trick, but this time wrapped in an AI-generated disguise.

Microsoft noted another clever move: the attackers used a self-addressed email tactic, where the sender and recipient were the same, with the real victims hidden in the BCC field. This technique helped bypass basic detection rules that look for mismatched sender-recipient pairs.

A Glimpse Into AI's Dark Side in Cybercrime

While the attack was relatively limited and primarily targeted U.S.-based organizations, it reflects a fast-growing trend: cybercriminals using AI to amplify their tactics.

"Like many transformative technologies, AI is being adopted by both defenders and cybercriminals," Microsoft warned.

On the defensive side, AI tools such as Microsoft Security Copilot help analysts detect and respond to threats faster than ever. But attackers are experimenting just as quickly—using AI to:

The result? A new wave of AI-assisted cyberattacks that are harder to identify and stop.

Why This Matters for Businesses

This campaign underscores a major shift in the cybersecurity landscape: AI is no longer just a tool for defenders—it's a weapon for attackers, too.

Even though the campaign's scale was small, it shows how easily generative AI can be weaponized to make old threats new again. As Microsoft points out, defenders must now learn to anticipate AI-driven threats and adapt just as quickly as the adversaries.

Organizations are urged to invest in AI-powered security monitoring, enhance employee awareness, and be wary of emails—even those that appear to come from familiar business accounts.