Your embeddings are not anonymous

There's a moment in every enterprise AI conversation where someone from legal, or security, or simply the person whose name is on the contract, asks the question that matters:

Where does our data actually go?

The answers that get offered are usually four, and three of them are weaker than they sound.

The four comfortable answers

"The provider doesn't train on our data." Usually true, and worth having in writing. It says nothing about who can read it, how long it's retained, which jurisdiction it sits in, or what happens in an incident.

"It's encrypted in transit." Table stakes, and irrelevant to the actual risk. Encryption protects the data from strangers on the network. It does nothing about the fact that the model, by design, receives it in plain text at the other end.

"The system prompt tells it not to reveal anything." We've written enough about this one. A prompt is not a boundary — not for permissions, and not against content designed to override it.

"We only store embeddings, not the documents." This is the one worth taking apart, because it's the most technical-sounding and the most widely believed.

An embedding is not a one-way door

When you prepare documents for an AI system, each passage is converted into a long list of numbers that captures its meaning. The original text is often discarded. What's left looks like noise, and it's tempting to treat that as a form of anonymisation.

It isn't. Storing only vectors prevents casual reading — nobody is going to browse your vector store and skim a contract. But it is not a cryptographic guarantee: given high-fidelity embeddings, a determined adversary can reconstruct a meaningful amount of the original text through model inversion techniques.

A vector store holds your customers' data. Not a hash of it, not a reference to it — a lossy but substantially recoverable copy of it.

Everything follows from accepting that sentence. Your vector database needs the same access controls as the database the documents came from. The same retention policy. The same answer to "which region is this in." The same entry in your data map when somebody asks what you hold about them.

And one thing more, which is the part that bites later: you cannot selectively delete from it. If personal data gets embedded, removing one person's information means rebuilding the index, because there is no row to delete — the information is distributed across vectors. Which means the only workable time to remove something is before it becomes a vector at all.

So: strip it on the way in

That leads to the single highest-value control in the whole pipeline, and it costs about a day to build: a cleaning step that runs before text is sent anywhere — before the prompt, before the embedding.

// Runs before the prompt is assembled, not after. Once a secret is inside a
// prompt it has already left the building — it is in the provider's logs, and
// if the text was embedded, inside a vector you cannot selectively delete.
func scrub(text string) (string, []Finding) {
    var found []Finding
    for _, rule := range rules { // card numbers, national IDs, keys, emails
        text = rule.Pattern.ReplaceAllStringFunc(text, func(m string) string {
            found = append(found, Finding{Kind: rule.Kind}) // the kind, never the value
            return rule.Token                                // "<CARD_REMOVED>"
        })
    }
    // Patterns miss novel key formats, so a high-entropy run of characters is
    // treated as a probable secret even when it matches nothing.
    return maskHighEntropyRuns(text), found
}

Two details matter more than the regular expressions. First, it records that something was found and what kind, never the value — otherwise your audit log becomes the leak. Second, it treats an unrecognised high-entropy string as a probable secret, because pattern lists only ever catch the key formats that existed when you wrote them.

Run it at ingestion, and run it again on anything a user pastes into a chat box. People paste credentials into assistants constantly, and they are not thinking about your retention policy when they do it.

The rest of the stack

Stripping secrets is one layer. Four others carry real weight.

Process in memory, keep nothing. Decrypt only in memory, never to disk; discard plaintext as soon as the embedding is generated; and strip request bodies from your observability pipeline so document contents don't quietly accumulate in your logging provider. That last one catches teams out constantly — the data was handled perfectly and then written to a log retained for ninety days.

Obfuscate the labels, not just the content. Filenames, folder paths and record identifiers are themselves disclosure. "Project Nightingale — termination clause" tells an observer most of what they need without a single line of the document.

Filter at query time, by the person asking. Every stored passage carries the identity and sensitivity of what it came from, and retrieval is constrained by the requester's access before results are ever assembled. This is the same forwarded-identity discipline we described in letting an agent act as you, pushed into the retrieval layer — and it's the control that stops one tenant's material reaching another, the failure we took apart in concurrency-safe isn't tenant-safe.

Decide what never leaves at all. Some material shouldn't reach a third-party model under any configuration. That's a real constraint and it has a real answer: a smaller model running inside your own boundary for those workloads, and the frontier model for everything else. Keeping the model behind a thin interface is what makes that a routing decision rather than a rewrite — the argument in swapping the brain.

What to ask, if you're buying

If you're the person being asked to approve one of these systems, six questions separate a considered design from an optimistic one:

• Where is the vector store, and who can read it? (If the answer treats it as less sensitive than the source documents, stop there.)
• What is stripped before text is sent to the model, and where does that run?
• How do you remove one person's data — and have you tried?
• Is plaintext ever written to disk or to logs?
• How does retrieval know what this particular user is allowed to see?
• Which workloads never go to a third-party model, and what runs them instead?

The honest position

None of this makes using a language model on sensitive data risk-free. It makes the risk explicit, bounded and reviewable, which is a different and achievable goal.

The failure mode worth avoiding isn't using these systems on real data. It's using them while believing a layer protects you that doesn't — and "it's only embeddings" is the most common version of that belief we encounter.