Encryption at rest and envelope encryption

Which adversary does each level of encryption stop, and what does it cost the storage layers?

Encryption at rest means data is stored in encrypted form on disk. Start with the correction, because it is the most consequential piece of folklore here.

Full-disk and volume encryption protect against physical media loss and nothing else in practice. The threat it addresses is a stolen drive, a decommissioned SSD, a backup tape, or a disk returned to a cloud provider under warranty. Against a compromised application, a stolen credential, a malicious insider with a login, or a runaway query, it does exactly zero. The operating system decrypts transparently for anyone the OS serves, and all of those adversaries arrive through the OS.

"Encrypted at rest: green" on a compliance dashboard is therefore a statement about disposal safety and a regulatory obligation. It is not a statement about your breach exposure. Both purposes are legitimate. Mixing them up produces false confidence. (This is what happened on Wei's dashboard in Why security is different.)

The granularity ladder

Moving to a finer unit of encryption narrows the adversary:

Per-column keys have a useful property. The platform operator can still run every maintenance job (compaction, which merges small files, and rewriting, which re-lays out files), because those jobs move bytes and do not need plaintext. The rule that generates the ladder is: the unit of decryption must be at least the unit of access. The granularity of protection is bounded below by the granularity of your key management. This is the granularity-mismatch law again: when key granularity differs from access granularity, you get over-decryption (more data is readable than the policy intended).

The price lands on the storage layers

Encryption is expensive in a way that is easy to miss.

• Ciphertext cannot be compressed. Good encryption output looks like random bytes (high entropy), and random bytes do not compress. So you must compress first, then encrypt. Even this order leaks: the length of the ciphertext reveals how compressible the plaintext was, which an attacker can turn into a way of reading secrets.
• Encryption destroys the structure that makes columnar storage fast. There is no dictionary encoding (replacing repeated values by short codes), no run-length encoding (storing "value x repeated 500 times"), no zone maps or min/max statistics (per-file summaries that let the engine skip files), no predicate pushdown and no partition pruning on an encrypted column.

So a per-column-encrypted table must be fully scanned and fully decrypted for every query that touches that column. That is exactly Wei's 40 minutes.

The tempting escape: encryption that preserves structure

Two schemes keep some structure so the database can still work:

• Deterministic encryption: the same plaintext always gives the same ciphertext, so equality joins and = filters still work.
• Order-preserving encryption: if a < b then Enc(a) < Enc(b), so range filters and sorting still work.

Both work, and both leak badly. Deterministic ciphertext preserves the frequency distribution. If you know the population (for example, that "O+" is the most common blood group), frequency analysis recovers values without ever touching the key. Order-preserving ciphertext preserves the ranking, and ranking plus any known distribution recovers most of the column. These attacks on property-preserving encryption have been demonstrated on real datasets. The general rule: preserved properties are published properties. Preserve equality and you publish the frequency distribution. Preserve order and you publish the ranking. Preserve format and you publish the domain size. Preserve linkability and you publish identity. This is an informational constraint, not a cryptographic one, so no better algorithm removes it.

Envelope encryption

Envelope encryption comes from a rate mismatch, not from a security requirement.

• The master key must never leave a hardened boundary: a hardware security module (HSM) or a managed key management service (KMS). This boundary can enforce policy, audit every use, and resist key extraction.
• But the boundary works at request rates (thousands per second) and millisecond latencies. Bulk data moves at GB per second. A system that called the key service for every block would run at the key service's speed.

So you split the work. A data key does the bulk encryption locally at full speed. The key service only ever sees the small (32-byte) data key, which it wraps (encrypts under the master key) and unwraps on request. The wrapped data key is stored next to the data, like a sealed envelope attached to a package.

The fixed cost of one key-service round trip sets the natural batch size: one data key per file, per partition or per table. This is the fixed cost sets the batch size law. And the batch is again the blast radius (the amount lost if that one key leaks). The same pairing appears in credential vending.

The structural consequence is more interesting than the speed. Once data keys are wrapped, the key service becomes the narrowest waist. Whoever can call unwrap can read the data, whatever the catalog's grants say. Encryption has not created an independent control. It has relocated the access decision into key policy. So this article builds on Enforcement points and does not replace it: encryption at rest is only as strong as the authorization on the unwrap call.

In one sentence: encryption collapses a data-protection problem into a key-policy problem, and destroys every structure the storage layers use to be fast.