AI Data Privacy: What Every Business Needs to Know

AI data privacy comes down to one question most teams never actually answer: when someone types something into an AI tool, where does it go? Free consumer tools and enterprise-grade ones handle that question completely differently, and the gap between them is where most AI privacy incidents happen, not a hack, just an unlucky default setting. Getting this right doesn't need a legal department or a six-figure compliance budget. It needs a clear picture of which tools your team uses, what tier they're on, and a short policy that says what's off-limits.

The short version: it's a tier problem, not a tool problem

Every major AI assistant, ChatGPT, Gemini, Claude, Copilot, splits its product into a consumer layer that often trains on your inputs by default, and a business layer that typically doesn't. OpenAI doesn't train on ChatGPT Team, Enterprise or API data by default. Microsoft 365 Copilot keeps your organisation's data within your Microsoft 365 tenant rather than using it to train the underlying models. Anthropic's Claude doesn't train on your conversations unless you opt in, and cut its API log retention from 30 days to 7 as of 14 September 2025. Google's Workspace AI tools follow a similar consumer-versus-business split. The rule of thumb: if nobody's paying for a business or enterprise tier, assume your inputs are fair game. None of this is exotic small print, either: it's on the vendors' own privacy pages, which is exactly where our tool directory pulls its ratings from, so you don't have to read five different policies to find the same three facts.

Do AI tools train on your data by default?

It depends which one, and which tier. Free and personal tiers of most major assistants may use your conversations to improve their models unless you actively opt out, a setting most people never find. Business and enterprise tiers, and API access, generally don't train on your data by default; that's the whole commercial pitch of paying for them. If nobody in your organisation can answer this question for the tools currently in use, that's the gap to close first, not a policy document. Worth a gut-check: ask two colleagues on different teams which tier they're actually signed up on. If they can't answer confidently, that uncertainty is the real privacy risk, more than any single vendor's policy wording.

What GDPR actually requires when AI touches personal data

If personal data of an EU or UK resident, a name, an email, anything that identifies someone, goes into an AI tool, GDPR applies whether you meant it to or not. You need a lawful basis for that processing, a Data Processing Agreement with the AI vendor, and a record of the activity. The ICO's guidance on AI and data protection is blunt about this: your organisation is on the hook for how personal data is handled inside an AI system, even one built by someone else. Plenty of teams do this without realising, every time a support agent pastes a customer's email into an AI drafting tool.

What HIPAA requires when AI touches health data

US healthcare teams face a sharper line. Any AI tool that processes protected health information needs a signed Business Associate Agreement, no exceptions, per HHS guidance on business associates. OpenAI, Anthropic and Google all offer BAAs, but only on their enterprise tiers, never the consumer ones. Using a personal ChatGPT or Gemini account to summarise a patient note isn't a grey area; it's a HIPAA violation the moment it happens, regardless of intent.

Does an AI tool keep a permanent record of what I type?

Usually yes, for some period, even on the safer tiers. Retention windows vary by vendor and product: Anthropic's Claude API now holds logs for 7 days rather than 30, business tiers of ChatGPT let admins set their own retention, and free consumer accounts often keep data far longer to support the training pipeline. 'Doesn't train on it' and 'doesn't keep it' are two different promises, check both before assuming a tool is safe for sensitive data. Ask specifically about backup copies too: a vendor can delete your active data on schedule and still hold it in backups for longer, which matters if you're ever asked to prove full erasure under a data subject request.

Where this usually goes wrong

A 20-person recruitment agency ran client CVs and interview notes through a free AI summarising tool for months, because it was fast and nobody had said not to. When a candidate exercised their GDPR right of access and asked what data the company held on them, the agency couldn't say what the AI vendor had retained or whether it had been used for training. Nothing malicious happened, no breach, no leak, but they had no answer to a routine request, which is its own kind of failure. One line in a policy, and one approved tool with a signed DPA, would have closed the gap. The awkward part wasn't a regulator knocking, it was a candidate asking a perfectly reasonable question and the agency having no answer, which is exactly the kind of gap a five-minute tool check would have closed months earlier.

What data should never go into an AI tool, on any tier?

Secrets and credentials, full stop, no tier makes those safe to paste anywhere. Beyond that, treat customer PII, health data and unreleased commercial plans as high-risk by default: only put them into a tool with a signed DPA or BAA and a confirmed no-training policy, never a personal or free account. When in doubt, the safe default is simple: if you wouldn't email it to a stranger, don't paste it into a consumer AI tool. Build this into onboarding rather than a one-off memo: new starters are the most likely to reach for whatever free tool a search engine suggests, simply because nobody's told them yet what the company's actual rule is.

Does deleting your account delete your data?

Not always, and not immediately. Most vendors will delete your account and stop active processing, but data already used in a completed training run generally can't be pulled back out of the model, and backups can persist for a defined window after deletion. If a tool has already trained on something sensitive before you deleted the account, the safest assumption is that the exposure already happened. That's the strongest argument for getting the tier right before you type anything sensitive, not after.

Check your tools, then write it down

Start by listing every AI tool your team actually uses, including the ones nobody officially approved; our guide on running an AI risk assessment walks through that in under a day. Then check each one's real data-handling profile, training, retention, DPA and BAA status, in our AI Tool Risk Directory rather than reading every privacy policy yourself. Finally, put the rule in writing with our free AI usage policy generator, so 'don't paste client data into personal ChatGPT' becomes a documented rule, not a hallway conversation.