Masking and tokenization

Why does masking the output not mask the predicate, and why is pseudonymization not anonymization?

Masking

Masking covers the case where the data must be visible to a person or system, but the sensitive value must not be. This is a different constraint from encryption: the reader holds no key, and nobody intends to reverse the transformation.

Static masking transforms the data at write time and produces a second dataset. The original cannot be recovered from it. Its real justification is that non-production environments (dev, test, training, analytics sandboxes, vendor demos) are the largest uncontrolled copy surface most organizations have. Each is a full copy with weaker controls than production. If you mask at the boundary, a leak there costs nothing.

The price is a second dataset: separate lineage (history of where data came from), staleness, and a masking job that must follow every schema change. If a new sensitive column is added and nobody adds it to the masking configuration, it passes through unmasked and silently. That silent failure, the new column nobody configured, is the most common way static masking breaks. (It is the same incentive gradient as the grant nobody revokes: the controls-decay law.)

Dynamic masking keeps one copy and transforms the value at read time according to policy. It is cleaner, and it inherits every problem of read-time row filtering from Enforcement points: it lives in the query plan, it constrains the optimizer and it costs pruning.

It also has a failure mode that row filtering does not, and it is the one to remember:

If the user can write WHERE national_id = <guess>, or join on the column, the returned row count leaks the value one bit at a time, at whatever rate the user can send queries.

-- Dashboard shows national_id as '***-**-1234'. The analyst runs:
SELECT count(*) FROM patients WHERE national_id = '123-45-6789';
-- 1 means the guess is right, 0 means wrong. Repeat with new guesses.

Display-only masking against a user who can filter is theatre (a show with no protection). A sound implementation must also restrict the operators allowed on the masked column: equality, ranges, joins, grouping, and any function that could move the value into the output. This is much harder to get right than the masking itself, and it is where most implementations are weak. This is Mateus's third finding.

Format-preserving encryption (FPE) exists because of a constraint that is neither physical nor cryptographic but organizational: downstream systems validate structure. A legacy application may require sixteen digits that pass a checksum, and it will not be changed. So the cipher must map the domain onto itself (digits to digits). But keeping the domain means the domain is small, and a cipher on a small domain is deterministic within that domain, so equality and frequency both survive. FPE buys compatibility with systems you cannot modify, and pays in exactly the leakage that makes deterministic encryption weak. It is a compatibility mechanism first and a security mechanism second, and it should be described that way.

In one sentence: masking hides the display while leaving the predicate open, and a filterable masked column is a searchable one.

Tokenization and pseudonymization

Tokenization replaces a value with a reference (a token) that has no computable relationship to the original value. The mapping from token to value is held in a vault, a protected database. Pseudonymization is the general idea of replacing direct identifiers with substitutes.

The difference from encryption is not strength. It is where the reversal happens.

• Decryption is offline, unobservable and unstoppable once the key is out.
• Detokenization is a service call. It can be authorized per request, rate-limited, logged, alerted on and revoked instantly. Encryption structurally cannot offer this.

If your threat model includes gradual insider exfiltration rather than one mass theft, tokenization gives you detection and throttling, where encryption gives you nothing, because with encryption there is no event to observe. This is why payment and healthcare systems moved to vaults rather than column encryption, even though a vault is an obvious architectural liability.

The liabilities are equally structural:

• The vault is on the hot path of every workload that needs real values. So it is an availability dependency and a scaling problem. You bought coordinated, auditable authority and paid with coupling.
• Its compromise is total, and unlike a key it cannot be rotated away: the mapping is the data.
• Tokens must be deterministic for the same real-world entity, or you cannot join tables, and joining is why you tokenized instead of deleting. Determinism preserves equality, so token frequency equals value frequency, and a token is a stable identifier across every system that holds it.

This gives the point people most often get wrong: pseudonymization is not anonymization. Under the EU's GDPR it is explicitly still personal data, because the means of re-identification exist and someone holds them. Replacing identifiers with tokens reduces exposure and supports data-minimization arguments, but it does not take the data out of legal scope. Building a compliance position on the opposite assumption is a common and expensive error.

In one sentence: tokenization moves reversal from an offline key to an online service, buying auditability and rate limiting at the cost of a hot-path vault that cannot be rotated.