AI Agents That Hack · Joseph Thacker

A hackbot is the term I’ve settled on using for AI systems with hacking abilities. I saw the term in the title of this talk from Blackhat 2023.

With the recent release of the paper LLM Agents can Autonomously Hack Websites (which has its flaws, but I digress) there is finally official research that agress with what I’ve been claiming for the past year. Current AI models can autonomously hack given the right setup.

In a previous post, I covered why I think hackbots will outperform humans at some point. This post will be a comprehensive discussion on hackbots. What can a hackbot do? Why do you care? Do they already exist? Etc.

Hackbot Defined

I define hackbot as any automated system that uses AI in a meaningful way in order to find vulnerabilities in hardened production applications (such as those with bug bounty programs). Almost all hackbots today are much more than a simple LLM. They are advanced systems which combine traditional code, existing tools, fuzzers, etc. with AI breakthroughs.

The ways that AI is used in hackbot systems varies a lot, but here are some ways I can see AI being useful:

Selecting which tool to run
Selecting which configurations to set for specific tools
Choosing which wordlists to use
Guiding the system for which features to hack on
Knowing which requests or parameters should be tested
Picking which payloads should be used for testing
Knowing whether the attacks were successful
Writing reports

Hero: My Hackbot POC

A few months ago, I wrote a little hackbot proof of concept. In order to inspire and help people see how a hackbot might work, I’ve decided to release it with this blog post. Here’s the link: https://github.com/jthack/hero

The project name hero is from Snowcrash, where the main character (who is a hacker) is named Hero.

It takes a raw http request as input, gives AI the ability to suggest potential vulnerabilities, write the payload, as well as execute and validate the findings. I hope you have fun playing with it or making your own!

Why Do You Care?

The hackbot industry is fascinating and relatively mind-blowing to me for a few reasons:

If humanity can build a decent hackbot, we could spin up thousands of them and find a massive percentage of vulnerabilities on the internet. That would help us secure it in an impactful way.
Beyond that, if hackbots work well, they can test code before it goes live, significantly lowering vulnerabilities that make it into production.
If a company can create a hackbot that can find bounties on bug bounty programs reliably, it’s basically a free-money machine until all the bugs it can find are gone.
If an enemy nation-state has a highly competent hackbot, that’s a serious national security issue.

Current Hackbots

So, do they already exist? I believe there are many variations on hackbots out there. Most of them are likely stealth mode start-ups or being developed by government agencies.

The four companies I’m aware of are:

I know a bit about each one except Sybil, and I can say that each of them are completely different from one another. They’re tackling the problem from different angles. They have different strengths and weaknesses. Their teams have varied amounts of expertise in AI, hacking, software development, etc.

I think there will be two or three times as many within six months.

My Dreams For The Future of Hackbots

I have a few hopes and dreams for the hackbot industry that I want to put out into the world.

To OpenAI and other foundation model makers: Please don’t nerf security expertise in your models. GPT4 is already denying me a good amount when using it for hacking. It’s not going to prevent this type of product. The builders will just move to open-source models.
To bug bounty platforms: What would it look like if you were to ask your hackers for permission to use their reports in training a hacking-specific model?
To all readers: This is pretty nuanced. LLMs by themselves can’t hack anything. LLMs in a system can hack some things. Over time they will be able to hack more. It’s hard to solve though.

Thanks,
Joseph

Source link