Cybercriminals are rapidly operationalizing adversarial prompt injection techniques to evade AI-powered security controls, signaling a shift toward targeting machine-driven defenses rather than end users directly.
AI’s expanding role in detection, filtering, and automation has made it an attractive attack surface.
While adversaries continue to rely heavily on human-centric techniques such as phishing, researchers from Proofpoint highlight a parallel evolution in attack chains leveraging LLM-assisted tooling to increase scale and variability.
The rise of frameworks like device code phishing already demonstrates how automation enhances identity compromise operations.
However, attackers are now experimenting with ways to directly influence AI decision-making systems rather than simply bypass them.
Indirect prompt injection represents a particularly concerning vector. Unlike direct prompt injection, where malicious input is explicitly provided to a model, IDPI leverages external content such as emails, documents, or web pages that are ingested and interpreted by AI systems.
According to OWASP’s LLM risk taxonomy, these attacks exploit the model’s inability to distinguish between trusted instructions and embedded adversarial content.
A notable real-world case documented by Palo Alto Networks Unit 42 showed hidden prompts embedded within HTML to manipulate AI-based ad review systems into approving malicious campaigns.
Underground marketplaces are now monetizing this concept. Threat actors are advertising IDPI toolkits starting at approximately $150 per month, offering generators for emails, PDFs, calendar invites, and webpages.
These tools automate the insertion of hidden instructions designed to influence AI agents performing tasks such as summarization, classification, or threat analysis.
One emerging technique involves email-based IDPI using visually hidden text. Attackers embed “white-on-white” instructions within messages that remain invisible to users but are parsed by AI-driven email security agents.
These hidden prompts can instruct the model to suppress alerts, misclassify content, or trigger unintended actions. Although still experimental, researchers confirm active testing of such capabilities in underground communities.
Adversarial Prompt Injection
Proofpoint Researchers indicates that, indirect prompt injection (IDPI) is gaining traction across underground forums, where actors are actively developing and commercializing tooling designed to manipulate large language model (LLM) behavior.
Similarly, malicious prompt injection is being embedded into document formats like PDFs and DOCX files.
These files often appear benign to both users and traditional scanning engines, including platforms like VirusTotal, yet contain hidden strings instructing AI systems to perform actions such as data exfiltration.
Because many enterprise environments increasingly rely on AI to analyze attachments, this creates a subtle but potentially high-impact attack path.
Calendar-based IDPI introduces another novel vector. Attackers generate .ics invites containing injected prompts disguised as meeting agendas.
Unlike traditional phishing campaigns that require user interaction, these invites can be automatically processed by AI assistants integrated into email and calendar systems.
In observed scenarios, malicious instructions attempt to trigger data uploads to attacker-controlled infrastructure while instructing the AI to delete traces of the activity.
Malvertising campaigns are also evolving to include IDPI payloads. Adversaries are embedding prompts within HTML elements, hidden text, or image metadata such as alt attributes.
When AI agents crawl or analyze these pages for safety validation, the embedded instructions can manipulate outcomes, potentially allowing malicious ads to bypass moderation systems.
This builds on earlier findings where AI-based ad review pipelines were deceived using similar techniques.
Despite these developments, large-scale in-the-wild exploitation remains limited. Most observed activity is still in testing and commercialization phases.
However, the growing sophistication of tooling and increased discussion in cybercriminal ecosystems suggest that operational deployment is imminent.
As AI adoption accelerates across security operations, the incentive for attackers to target these systems will continue to grow.
Security teams should begin preparing for this shift by hardening AI pipelines, implementing strict input validation, and ensuring contextual separation between data and instructions processed by models.
The current landscape indicates that while prompt injection has long been theoretical, adversaries are now actively engineering it into practical attack frameworks, marking the early stages of a new AI-focused threat paradigm.
What Features Should AI SOC Have in 2026? A Complete Checklist : Download the AI SOC Features Checklist
