Hackers Can Weaponize AI Summarizers to Spread Malware

A new cybersecurity research has exposed a concerning way in which hackers can use AI summarizers to spread ransomware attacks.

Researchers at CloudSek explain that via a ClickFix social engineering attack, hackers can embed harmful instructions within documents using invisible coding techniques. The hidden text remains invisible to human eyes but AI summarizers can detect it, and when they generate summaries, they may unknowingly pass on the dangerous instructions to users.

“A novel adaptation of the ClickFix social engineering technique has been identified, leveraging invisible prompt injection to weaponize AI summarization systems,” researchers said.

The attack depends on CSS-based obfuscation which uses zero-width characters and white-on-white text and tiny fonts and off-screen placement to hide instructions. The summarizer detects the invisible code which remains invisible to human eyes.

The attackers implement prompt overdose by adding numerous commands to hidden sections which causes the AI to select these commands first in its output.

“When such crafted content is indexed, shared, or emailed, any automated summarization process that ingests it will produce summaries containing attacker-controlled ClickFix instructions,” the study explained.

During testing, researchers demonstrated how an AI summarizer could be manipulated to instruct users to run dangerous PowerShell commands.  While the test version was harmless, a real attack could easily launch ransomware.

The risk level of this attack is high because AI summarizers are integrated into email clients and browsers and workplace applications. The researchers point out that many people trust the summaries without reading the original document, making it easier for attackers to exploit that trust.

Experts recommend defenses such as stripping out hidden text before content is summarized, filtering suspicious prompts, and warning users if summaries contain step-by-step instructions.

Without these protections, AI could unintentionally become “an active participant in the social engineering chain.”