Users lack control as major AI platforms share personal info with third parties

Some of the most popular generative AI and large language model (LLM) platforms, from companies like Meta, Google, and Microsoft, are collecting sensitive data and sharing it with unknown third parties, leaving users with limited transparency and virtually no control over how their information is stored, used, or shared, according to Incogni.

AI platforms trap user data in training

Many of these platforms, including Google’s Gemini, Meta AI, DeepSeek, and Pi.ai, do not appear to offer ways to opt out of having their prompts used to train AI models. Once personal or sensitive data is entered, there is no practical mechanism to delete it from an AI model’s training dataset.

While laws like the GDPR grant individuals the right to request data erasure, it’s still unclear how to practically remove the information from a machine learning model. As a result, many companies are not currently obligated, or technically able, to remove such data after the fact. Contact details or proprietary business details may become embedded in the model’s training data, potentially without the user’s explicit knowledge or consent.

Who collects what

As generative AI becomes a growing part of everyday life, users are often unaware of what personal data these tools collect, how it’s used, and where it ends up. Researchers analyzed leading AI platforms across 11 subcategories in three key areas: how user data is utilized in model training, the transparency of each platform’s privacy practices, and the scope of data collection and third-party sharing.

• Meta.ai and Gemini collect precise location data and physical addresses of their users.
• Claude shares email addresses, phone numbers, and app interaction data with third parties, according to its Google Play Store listing.
• Grok (xAI) may share photos provided by users and app interactions with third parties.
• Meta.ai shares names, email addresses, and phone numbers with external entities, including research partners and corporate group members.
• Microsoft’s privacy policy implies that user prompts may be shared with third parties involved in online advertising or using Microsoft’s ad tech.
• Gemini, DeepSeek, Pi.ai and Meta.ai, most likely are not giving users the ability to opt out of training the models with their prompts.
• ChatGPT turned out to be the most transparent when it comes to the information on what prompts will be used for model training, and a clear privacy policy.

Even for users seeking clarity, the details are often buried in fragmented help pages or written in dense legalese. Incogni found that every analyzed privacy policy requires a college-level reading ability to interpret.

Employee use of AI could leak confidential business information

In addition to individual privacy, businesses may face even greater risks. Employees frequently use generative AI tools to help draft internal reports or communications, not realizing that this can result in proprietary data becoming part of the model’s training dataset. This lack of safeguards not only exposes individuals to unwanted data sharing but could also lead to sensitive business data being reused in future interactions with other users, creating privacy, compliance, and competitive risks.

“Most people assume they’re chatting with a trusted assistant, and not giving away their contact details or confidential business information,” said Darius Belejevas, Head of Incogni. “The reality is far more invasive, and companies don’t make it easy to understand what’s really happening with your data. Users deserve to know what’s being collected, who’s seeing it, and how to stop it. Right now, those answers are often hard to find, or don’t exist at all.”