The conversation about AI and confidential data usually stalls on the wrong question. “Will they train on our data?” has a documented answer, it has been no by default on both major US providers for a long time, and it is written into their terms. Meanwhile the paths by which your data actually escapes are somewhere else entirely, and nobody is looking at them.
What the published terms actually say
Start with the floor, because it is better than most people assume.
OpenAI states that data sent to its API is not used to train or improve its models unless you explicitly opt in, a position in force since March 2023. Anthropic commits that retained data is never used for model training without express permission, and that conversation content is not retained by default on the Claude API.
Default retention exists for abuse monitoring. OpenAI retains API inputs and outputs for up to 30 days to provide the service and identify abuse, then removes them unless legally required to keep them. Anthropic’s equivalent default is no retention of prompts and outputs, with a specific exception: certain designated models require 30-day retention as a condition of access.
Above that floor sits zero data retention, which both offer to qualifying customers. Under Anthropic’s arrangement, prompts and responses are not stored at rest once the API response is returned. OpenAI’s excludes customer content from abuse monitoring logs and forces request storage off even if a request asks for it.
Everything above describes the direct APIs. On Amazon Bedrock and Google Cloud’s Agent Platform the cloud provider is the data processor rather than the model vendor, so the retention terms that apply are the cloud provider’s, not the ones on the model vendor’s site. If your data protection assessment cites the wrong company’s policy, it is describing a system you are not using.
Three exceptions that matter more than the headline
These are all documented. They are also all easy to miss, and each one has caught a team that believed it had solved this.
Flagged content survives your arrangement
Even with zero data retention in place, a provider may retain data where required by law or where automated trust and safety systems flag it. Anthropic documents that a flagged session may have inputs and outputs retained for up to two years. OpenAI retains flagged image and file inputs for manual review regardless of zero data retention status.
What to do: this is not a reason to avoid the API, it is a reason to know it before you sign a data processing agreement, and to make sure legal knows it too. An arrangement described internally as “they keep nothing” is inaccurate, and inaccurate in the direction that matters during an audit.
Stateful features are outside it, and nothing blocks you
Zero data retention covers the stateless inference path. Features that store jobs, files, container state or session transcripts cannot be zero-retention by definition. Code execution containers retain data for up to 30 days. OpenAI’s assistant and thread endpoints hold application state indefinitely even for approved zero-retention customers.
The sharp edge is in Anthropic’s own documentation: under a zero-retention arrangement the API does not block these features. Using one is a choice to step outside your arrangement for that data, and the feature’s own retention policy applies instead.
What to do: that choice is made by whoever writes the next feature, possibly months later, without reading the contract. Put it in a code review rule, not a policy document.
The best model may be the one you cannot have
Some frontier models are designated as requiring 30-day retention as a condition of use, and zero data retention is not available for them. On the Claude API a request to one of those models from an organisation configured for zero retention returns an error rather than silently complying.
What to do: check model eligibility at the point you choose a model, not after a proof of concept has been built on one you cannot deploy. Anthropic supports enabling 30-day retention for a single workspace, so a hybrid split is possible if the use case justifies it.
The leak nobody negotiates: your own retrieval layer
Here is the failure we see most often, and it has nothing to do with the model vendor.
A team indexes the company’s documents into a vector store so an assistant can answer questions over them. The index contains the HR folder, the board pack and the salary review, because the crawler had broad credentials and nobody scoped it. A junior employee asks a plausible question. Retrieval finds the most semantically similar passages, the model faithfully summarises what it was given, and confidential material arrives in a chat window.
No provider term was breached. Nothing was trained on. The model behaved correctly. The permissions were enforced in the source systems and thrown away at indexing time.
The fix is to filter by entitlement at query time, inside the vector search, rather than retrieving broadly and dropping results afterwards. Post-filtering is worse than it sounds: it hides the leak from your logs while leaving it in the retrieved set, and it quietly degrades answer quality because the model never sees the passages it was allowed to see.
Two more of the same family, both routinely missed:
- Tracing tools hold everything. Whatever you use to debug prompts stores the full assembled prompt, which includes every retrieved passage. That is a complete copy of your confidential text in a third-party product with its own retention policy, usually a more generous one than your model provider’s.
- Schemas and tool definitions are not always covered. Anthropic caches the JSON schema used for structured outputs for up to 24 hours and states explicitly that protected health information must not be placed in schema definitions. Field names and enum values are data too.
The three architectures, and what each one actually buys
Managed API, under contract
- Fastest to production and the best models
- Protection is contractual, with the documented exceptions above
- Cross-origin requests may be unsupported, so calls route through your backend
- Right for most internal tools and most commercial data
Your own cloud tenancy
- Inference inside the cloud account you already hold
- Your cloud provider becomes the data processor, under an agreement you likely already have
- Fewer new vendors for procurement to assess
- Right when the blocker is legal rather than technical
Self-hosted open weights
- Nothing leaves your network, verifiable rather than promised
- You now own GPUs, patching, scaling and evaluation
- A real capability gap against frontier models, narrowing but present
- Right for regulated workloads and genuinely secret data
Most organisations that believe they need the third column need the second. The blocker is usually a data processing agreement and a residency requirement, both of which the cloud route answers without taking on a GPU fleet. Running your own models is a legitimate choice, but it is an infrastructure commitment, and it lands in the same budget as everything else in what it costs to run an LLM feature in production.
A cheaper move than any of the three: send less. Redact identifiers before the prompt is assembled, retrieve passages rather than whole documents, and drop fields the model does not need to answer the question. Every control above is about protecting data in transit and at rest. Not sending it is the only one that cannot fail.
What to settle before you build
- Which arrangement are we actually under? Default retention, zero data retention or a cloud tenancy. Name it, and name the company that is the data processor under it.
- Which models are eligible under it? If the model your prototype used is not, find out now.
- Which endpoints are we allowed to call? Stateful features sit outside zero-retention arrangements and are not blocked. This belongs in code review.
- Does retrieval filter by the asking user’s entitlements, in the query? If the answer involves filtering results afterwards, it is not a yes.
- Where do traces go, and for how long? Including the ones a developer enabled for a week and forgot.
- What happens when content is flagged? Up to two years of retention, outside your arrangement. Legal should hear this from you, early.
None of this is a reason to keep AI away from your own data. It is the difference between a feature that survives a security review and one that gets switched off after it. The choice of approach also interacts with the architecture, which we cover in RAG, fine-tuning or prompting, and with whether the use case was worth doing at all, in where AI actually pays off.
Our AI and machine learning team builds these systems with entitlement-aware retrieval and the retention position written down before the first prompt is sent. If you have confidential data and a use case you have been told is impossible, tell us what the constraint is. It is often narrower than it was described.
Common questions
Do AI providers train on data sent through their APIs?
Not by default. OpenAI states API data is not used to train or improve its models unless you explicitly opt in, in force since March 2023, and Anthropic commits that retained data is never used for training without express permission. Default retention for abuse monitoring is a separate matter: OpenAI keeps API inputs and outputs for up to 30 days, and certain designated models require 30-day retention as a condition of access.
What does zero data retention actually cover?
The stateless inference path. Prompts and responses are not stored at rest once the response is returned, and are excluded from abuse monitoring logs. It does not cover stateful features such as code execution, file storage or assistant threads, and those are not blocked, so calling one steps outside the arrangement for that data. It also does not override legal holds or automated safety flags, which can result in retention for up to two years.
How do you stop an AI assistant leaking confidential documents?
Filter retrieval by the asking user’s entitlements inside the vector query rather than dropping results afterwards, scope what gets indexed in the first place, redact identifiers before the prompt is assembled, and check what your tracing tool retains. The most common leak is an index built with broad credentials that discarded the permissions enforced in the source systems.
Should we self-host an open-weights model for privacy?
Only if the requirement is genuinely that nothing leaves your network. Self-hosting means owning GPUs, patching, scaling and evaluation, against a capability gap that is narrowing but real. Running inference inside your own cloud tenancy answers most legal and residency objections without that commitment, because your existing cloud provider becomes the data processor.
Sources
- Anthropic, API and data retention documentation, retrieved September 2026: conversation content not retained by default. Retained data never used for training without express permission. Zero data retention means prompts and responses are not stored at rest after the response is returned. Designated models require 30-day retention and are not available under zero data retention. Stateful features are ineligible and are not blocked by the API. Code execution container data retained up to 30 days. Structured output schemas cached up to 24 hours. Cross-origin requests unsupported under zero data retention. Data flagged by automated trust and safety systems, or subject to legal hold, may be retained for up to two years regardless of arrangement.
- OpenAI, API data controls documentation, retrieved September 2026: API data not used to train or improve models unless explicitly opted in, since 1 March 2023. Abuse monitoring logs retained up to 30 days by default. Zero data retention excludes customer content from those logs and forces request storage off. Assistant and thread endpoints retain application state despite zero-retention approval. Image and file inputs are scanned on submission and flagged content is retained for manual review regardless.
- On Amazon Bedrock and Google Cloud’s Agent Platform the cloud provider acts as data processor rather than the model vendor, per Anthropic’s own documentation, which directs customers to those platforms’ retention and compliance terms.
- Provider terms change. Verify the current position against the provider’s own documentation and your contract before relying on any of the above.
Written September 2026.
