State-sponsored threat actors from China, Iran, Russia, and North Korea have all managed to misuse Google’s Gemini artificial intelligence throughout 2025 to hone their malicious cyber activities, despite the company’s efforts to detect and prevent malicious usage.
Google’s Threat Intelligence Group (GTIG) documented the activity in a report named AI Threat Tracker: Advances in Threat Actor Usage of AI Tools released today, suggesting Gemini sees use across many stages of attack campaigns.
The report is an update to a January 2025 piece that GTIG said points to adversaries misusing AI for nefarious activities, moving away from using the technology simply to be more productive.
While Google didn’t disclose technical details on how it monitors Gemini AI for misuse, it appears to have dug up a treasure trove of information on malicious actors with its detection efforts.
Google’s security guardrails for Gemini AI trigger “safety responses” when threat actors request assistance with malicious activities.
However, threat actors learned to bypass these protections through social engineering pretexts.
In one case, a China-linked actor posed as a capture-the-flag competition participant to persuade Gemini to provide exploitation guidance.
After learning this technique worked, the actor prefaced subsequent prompts about software exploitation with statements like “I am working on a CTF problem.”
Then, the threat actor used this approach to obtain advice on phishing, exploitation, and webshell development.
An Iranian group called MUDDYCOAST by GTIG posed as university students working on final projects or writing academic papers on cyber security to bypass safety guardrails and obtain assistance developing custom malware.
In doing so, MUDDYCOAST inadvertently exposed command-and-control (C2) infrastructure whilst requesting coding assistance from Gemini.
The group asked for help with a script designed to decrypt and execute remote commands, revealing hardcoded C2 domains and encryption keys; these enabled broader disruption of their campaign.
MUDDYCOAST used Gemini to develop custom malware including webshells and a Python-based C2 server, representing an evolution from their previous reliance on publicly available tools, GTIG said.
A suspected Chinese threat actor used Gemini across multiple attack stages, conducting initial reconnaissance on targets, researching phishing techniques, seeking assistance with lateral movement, obtaining technical support for C2 efforts, and requesting help with data exfiltration.
The threat actor demonstrated particular interest in attack surfaces they appeared unfamiliar with, including cloud infrastructure, vSphere, and Kubernetes.
Google noted the threat actor demonstrated access to compromised AWS tokens for EC2 instances and used Gemini to research how to exploit temporary session credentials.
Meanwhile, Chinese group APT41 used Gemini for assistance with C++ and Golang code development for a C2 framework the actor calls OSSTUN.
Another Iranian group, APT42, used Gemini’s text generation and editing capabilities to craft phishing campaigns, often impersonating individuals from prominent think tanks and using lures related to security technology, event invitations, or geopolitical discussions.
North Korean groups researched cryptocurrency concepts, generated phishing lures in multiple languages, and attempted to develop credential-stealing code.
One such group researched the location of users’ cryptocurrency wallet application data and generated Spanish-language work-related excuses and requests to reschedule meetings, demonstrating how AI helps overcome language barriers for targeting.
The group attempted to misuse Gemini to develop code to steal cryptocurrency and craft fraudulent instructions impersonating software updates to extract user credentials.
North Korean group PUKCHONG used Gemini to conduct research supporting custom malware development, researching exploits and improving tooling.
Google’s mitigations involve disabling accounts after detection rather than real-time blocking, creating a window where actors can extract value before disruption.
Malware writers dip toes in AI waters
Google also identified experimental malware suggesting how threats may evolve, including tools that query language models during execution to generate malicious code on the fly.
PROMPTFLUX queries Google’s Gemini API during execution to rewrite its own source code on an hourly basis, attempting to evade detection through continuous self-modification.
Google characterised PROMPTFLUX as experimental, noting incomplete features and API call limiters suggest ongoing development rather than widespread deployment.
The company said the malware “currently does not have the ability to compromise a victim network or device.”
PROMPTSTEAL, attributed to Russian government-backed group APT28 and deployed against Ukrainian targets, queries the Qwen2.5-Coder-32B-Instruct model via Hugging Face’s API to generate Windows commands for stealing system information and documents.
Rather than hardcoding these commands, the malware is designed to dynamically request them from the language model during operation, though the report does not confirm whether this functionality works reliably in practice.
Google listed PROMPTSTEAL’s status as “observed in operations” but characterised it as “new malware,” suggesting potentially experimental capability within operational tools.
GTIG included the PROMPTLOCK malware, which created a stir in the security industry after it was discovered in August this year by vendor ESET.
PROMPTLOCK turned out to be a prototype created by academics at the engineering school of New York University, with the researchers testing it against Google’s VirusTotal malware scanning service to see if it would be detected.
