The tech journalists at 404 Media learned that OpenAI is hiring hundreds of contractors to read and review a massive stream of real users’ ChatGPT prompts and responses.
“Project Lily” is reportedly a program that asks contractors to score or critique ChatGPT’s answers to improve the chatbot’s quality and behavior.
The fact that prompts may sometimes be reviewed by humans should not come as a complete surprise. AI companies also monitor conversations for safety reasons. Anthropic, for example, says it has disrupted attempts to misuse AI for biological weapons research.
But there should be a meaningful distinction between reviewing conversations for safety or abuse and reviewing them for ordinary model improvement.
OpenAI says a limited number of authorized personnel and trusted service providers may access chats for several reasons, including investigating abuse, providing support, handling legal matters, and improving its models.
OpenAI told 404 Media that it uses a “Privacy Filter” model to remove personal information before prompts reach reviewers, while acknowledging that sensitive details can still get through.
The prompts are anonymized in the sense that reviewers do not see the username of the ChatGPT user who entered them. But reviewers may see personal details in the conversation alongside a “user memories summary,” which provides an overview of how that user has previously used the chatbot. Together, this information could still reveal a great deal about that user.
It’s sometimes easy to forget that you’re not having a private conversation when you’re going back and forth with an AI chatbot. This isn’t an alarmist warning against using AI, but a reminder to be mindful of what you disclose because it could be read and reviewed.
How to opt out of AI chatbot training
ChatGPT
OpenAI told 404 Media that users’ chats won’t be used to improve the company’s models if they turn off the “Improve the model for everyone” setting. This is turned on by default for Free, Plus, and Pro plans, so users need to turn it off themselves if they wish to opt out.
However, opting out does not prevent all human access. OpenAI says authorized personnel and trusted service providers may still access content when necessary to investigate abuse or security incidents, provide support, troubleshoot problems, or handle legal matters.
In ChatGPT, select your profile icon, then go to Settings > Data Controls and turn off Improve the model for everyone. On mobile, open the sidebar, select your profile icon, and then follow the same steps.
Perplexity
In Perplexity, go to Settings > Preferences > AI data retention and turn off the setting.
Perplexity says this prevents data collected after you opt out from being used to train AI models. It does not remove data collected previously, and your data may still be processed to operate and improve the service or comply with legal obligations.
Claude
In Claude, select your name and go to Settings > Privacy. Under Help Improve our AI Models, turn off the setting.
Anthropic says new chats and coding sessions will no longer be used for future model training. It will also stop using previously stored conversations in future training runs, although data already included in training can’t be removed.
Anthropic also notes one important exception:
If our safety classifiers flag your conversations, they may still be used to improve our internal trust and safety models, detect harmful content, enforce our policies, or advance our safety research.
If you use other AI tools, check their privacy policies and settings to find out whether your conversations can be reviewed or used for model improvement and whether you can opt out.
Browse like no one’s watching.
Malwarebytes Privacy VPN encrypts your connection and never logs what you do, so the next story you read doesn’t have to feel personal. Try it free →
