How AI Defenders Will Protect Us From Manipulation

One of the AI topics that I’ve been talking about for the last few months is Context. The basic argument is that everything we’re about to do with AI will sit on top of a deep, nuanced understanding of the principal—which could be an individual, a business, or whatever.

Here are some examples of where AI is much more powerful when it knows about the subject it’s helping.

• 🗣️Therapy — You ask an AI assistant why you’re feeling sad, or what you can do to feel better. It can do a much better job if it knows your background, your history, has exposure to your journal, your goals as an individual, your life challenges, your work and financial situation, etc.

• 🧳Work — You ask an AI assistant to help you solve a problem at work. It can do a much better job if it knows the company’s capabilities, the details around the challenge, the resources we have available to solve it, etc.

• ✍️Writing — You ask an AI assistant to help you write a story or a screenplay. It can do a much better job if it knows the types of stories that interest you, based on seeing your ratings of other films, or having access to your favorite books or movies.

These are just a couple examples and the list is infinite. It’s really any situation where problem solving is improved by more deeply understanding the problem, and that’s almost always.

You could of course try to jam a bunch of context into each request. So when you go to ask for a story you try to feed it a bunch of stuff you like. But that’s annoying. Plus you won’t remember everything in the moment. Plus it’ll be too much to remember and add each time.

The better, obvious, and inevitable solution is that your AI assistant will simply maintain continuous context about you, across multiple dimensions, and it’ll keep it updated with incoming data. Your workouts, mood ratings, your diet, journal entries, etc. People will be hesitant to share for the first few years, but soon our AI assistants will basically have everything.

And while that will be utterly awesome functionality-wise, it’ll also present an unprecedented attack surface as well.

We’re not talking about physical attacks here; we’re talking about people being tricked, manipulated, duped, and otherwise convinced or coerced into doing something they wouldn’t want to.

Context is a tool, and many tools can be used as weapons. Context is especially dangerous as a weapon when it’s an attacker using it to attack the principal. Let’s open our minds up to not just 1-on-1 attacks but groups and organizations against groups or individuals.

• ✍️AI Assistant Data Brokers — We already have Data Brokers who collect and sell way more data than the Dark Web could hope to have. They do that for marketing purposes, but once everyone has these rich AI profiles they are going to become targets for not just “legit” Data Brokers, but underground markets who collect that data on high-value targets.

  Imagine a service where you can find crypto holders, or people bragging about how much money they make, or posting pictures of their opulent vacations. Now gather all their Context via a hacked Personal AI assistant profile or some OSINT/Recon. Now that information is for sale.

• 🗣️Propaganda Attacks — Both special interests (corporate, activist, or whatever) and governments can also use this type of information to target people or groups with specific campaigns. They might not need that extra context, but the more targeted they go the more they can tailor the messaging to that particular mark. Think: changing opinions on political events, destroying the reputation of their enemies, etc.

• 💰Marketing — Marketers will happily purchase this data, or collect it themselves however they can, to use the same propaganda techniques to make people aware of their space, their product/service, or whatever. They’ll be able to slowly and effectively drive behavior in a way that benefits them.

• 🕵️Feelings Hacks — Perhaps most scary is what will be possible when social engineers get access to this information, especially when they’re already experts at pressing buttons. But now instead of cue-reading outside buttons, and stuff they get from purely public information, they’ll be able to tailor their attacks to a target’s background, history, trauma, and other highly-revealing information.

This is the type of wedge that cuts into people’s inner circles, and it will all be AI powered as well. E.g.,

Given this Target Context, construct the ultimate entry script for our new recruits going after this target.

• 🖤AI-powered Pig Butchering — Once place this will do extraordinary damage is with Pig Butchering attacks, which is where attackers use companionship and/or romance to get lonely (often elderly) people to part with their money. They often play out over months as the attacker gains trust with the target. Then at the end they take whatever the target has and disappear.

  This type of attack will be a lot more effective, and even automatable, using the combination of Context and AI Agents.

And now we arrive at the point of the article.

All this was buildup to say that Context will soon be wielded against us to:

• Get us to believe things

• Get us to think things

• Get us to buy things

• Get us to feel things

And ultimately, to control us. The scariest part of it is that because these are hidden buttons being pressed, and AI will be doing a lot of the campaign creation, the target often won’t even know it’s happening.

But you know who will know? Your AI Defender. It has the most Context of all.

I know. I saw your face crinkle up. You’re thinking:

Yeah. Unfortunately. This isn’t what I’m prescribing. It’s what I think is coming, and there’s not anything anyone can do about it.

The way this AI Defender will work is actually pretty simple. It’s just cat and mouse, and mouse and cat, round and round.

Basically your AI Defender (just another personality of your AI Assistant) will be in charge of defending you. And it knows you better than anyone, including you.

So when you meet someone cute who starts flirting and looking at your clothes, and starts complimenting you, and maybe mentions a shared piece of background, it’ll start engaging to defend you.

Same for buying products.

Same for political opinions.

Basically our AI Assistant will know what pushes our buttons, because it’s the world’s expert on those buttons.

It will also see our behavior, and will be able to see if that behavior is tracking with the desires of the propaganda/manipulation we’re being exposed to.

And it can warn us, prompt us, and otherwise pull us out of the tunnel that the manipulator is trying to take us down.

This will be an upcoming post, but now realize that your AI Assistant/Defender will also have edit capabilities. What happens when it can:

• Remove the label from products

• Remove manipulative language from writing

• Overwrite edit incoming audio that would press buttons

Terrifying as hell? Absolutely.

Imagine attackers/governments getting access to that interface? Even worse, they won’t have to hack it. They’ll pay people to use their filters.

1. Manipulators work by pushing buttons

2. Deep Context will make AI assistants infinitely more powerful, but that same context will get used as intimate buttons by attackers

3. People will constantly be under attack by AI powered systems abusing their Context

4. Paradoxically, our AI Defenders will monitor that 24/7 and let us know when it’s happening

5. The next step after that is prophylactic controls, i.e., filtering the attacks from even hitting you, which will also be used against us