ComputerWeekly

Threat actor use of AI becoming routine, warns Google


Threat actor use of AI in malicious cyber campaigns is evolving at pace, with several trends emerging over the past three months that point to a fast-approaching inflection point, according to new research published this week by the Google Threat Intelligence Group (GTIG) and Mandiant.

Google’s team said that while previously it had only observed “nascent adversary interest” in agentic capabilities, since the summer multiple threat actors have started to abuse open source, agentic pen-testing frameworks to automate elements of their intrusions.

“Threat actor adoption of these frameworks represents a paradigm shift towards intrusions occurring at machine speed, with the automated chaining of complex attack paths scaling operations significantly,” the team wrote in its report.

“While the mechanics of the intrusions remain the same, such as using compromised credentials and targeting unpatched systems, the speed and scale at which frameworks can exploit these known weaknesses introduces a new defensive challenge.”

Some of the more popular open source frameworks being turned to malicious ends include Artex – recently the subject of an FBI advisory – Cairn, CyberStrikeAI and PentAGI, said Google. It said it had seen Cairn used in several attacks against targets in the aerospace sector in the US, while PentAGI has been spotted being used for recon and vulnerability scanning in cyber attacks against Asian organisations over the past couple of months.

Google said that both financially motivated cyber criminal gangs and nation-state backed threat actors were likely using these frameworks, and said that their further use was a virtual certainty. Moreover, said the team, “As the sophistication of these tools and underlying LLMs increases, we anticipate not only reduced human involvement in these operations but also increased stealth and sophistication of the activity.”

At the same time, threat actors are also using multi-agent systems and LLM-driven pipeline to automate the more labour-intensive parts of vulnerability analysis, including tearing down patches and weaponising proof-of-concept (PoC) exploits. This is a grave concern because in doing so, they can cut the window between supplier patch disclosure and in-the-wild exploitation.

The Google team suspects threat actors have realised it is more efficient to use LLMs to weaponise patched n-day flaws instead of uncovering new zero-days because they can zoom in on a finite set of source materials, compare differing source code versions, and examine PoC code, release notes, security research and even media coverage. In contrast, zero-day hunting requires them to cast a much wider net.

The team said this was evidenced in the fact that while disclosures and exploits are skyrocketing – as seen in Microsoft’s monthly Patch Tuesday updates – the number of zero-days has risen only marginally.

“LLMs are transforming how vulnerabilities are discovered and exploited across public and enterprise infrastructure, shrinking the patch window for organisations. Simultaneously, the lower barrier to entry has fuelled a wave of LLM-generated exploits that are claimed to work but don’t, creating additional verification work for defenders,” said Google.

And financially-motivated threat actors are increasingly profiting from so-called LLMjacking – the theft of LLM credentials and cloud resources from compromised environments and their subsequent resell through underground proxies. This activity ultimately enables malicious groups to overcome the sort of problems, such as access costs, token rate limits and geographic restrictions, that often dog legitimate LLM usage.

Google said such access was in high demand, and it had observed the underground cyber criminal marketplace moving to meet it, for example by transitioning beyond the sale of mass generic identity dumps to placing a premium on verified frontier AI accounts and tokenised API quotas as turnkey utilities.

Mandiant said it had responded to several LLMjacking incidents in the past few weeks in which threat actors were racking up fraudulent charges exceeding $100,000 a day, and as high as $1m in some instances.

Information operations

Meanwhile, threat actors engaged in information operations (IO) – which can include state-sponsored espionage groups, inauthentic political monetisation (IPM) groups, and commercial organisations conducting influence-as-a-service (IaaS) operations such as private intelligence contractors and even unethical marketing and PR agencies, are leaning into full operational LLM integration, said Google.

Such groups routinely use LLMs now to produce synthetic media, clickbait and other forms of AI slop, as well as harnessing them to automate domain network workflows, for example SEO manipulation, editing and formatting of seeded articles, and automated generation and publishing of malicious information.

In August, Google said it observed an unnamed group using its Gemini API alongside tools from others to build an end-to-end IaaS platform to manipulate public opinion in Europe and the Middle East.

The platform was engineered to automate the entire IO lifecycle, from the creation of sockpuppet personas through media generation and distribution and automated posting on platforms. Analysis by the GTIG team found the threat actor’s prompts were focused on emulating organic grassroots engagement while evading detections and mitigations put in place by social media platforms.



Source link