Cost as a substitutable resource
Money buys position on every other axis; the only axis denominated in a different unit.
Where you meet it: provisioned vs. serverless, spot vs. on-demand, replication factor, precompute spend, retention depth.
What it is
"Object storage is cheap" is true per byte stored and false per operation (request pricing). At about $0.0004 per thousand GETs, a billion small reads cost 10⁹ ÷ 1,000 × $0.0004 = $400 in requests alone, which can exceed the storage bill. Your row-group size is partly a billing decision. The fixed cost in the batch-size rule is, literally, denominated in dollars here, not only in milliseconds. This is the first time here that the constraint is economic and not physical.
From Storage: Physical Organization › Object storage and the lakehouse
Compression is the clearest case of money buying position on another axis. Cores can be bought, and compression turns bought cores into fewer bytes at a computable rate, but the cores and the bytes come from different budgets.
From Encoding, Serialization & Compression › What compression forbids
Where it appears
In 9 mechanisms. Each says what the mechanism is about, then how this trade-off shows up there, in its own words.
- Storage: Physical Organization
What should sit next to what, given that I must decide at write time and find out at read time?
Your row-group size is partly a billing decision.
Read it inObject storage and the lakehouse
- Encoding, Serialization & Compression
What do we turn our values into when we turn them into bytes, and what does that choice cost?
That shift is an economic accident, not a permanent rule.
Read it inBytes become an outputWhat general-purpose compression forbids
- Memory & Resource Management
Who is allowed to hold how much memory, for how long, and what happens to the work when the answer is less than it asked for?
Residency is ultimately an economic decision with a computable break-even.
Read it inTwo live claims: mmap and cheap RAM
- Caching
What happens when we deliberately keep a second copy of data that no protocol forces to be correct?
It buys pooled capacity (+ln k / ln N), recompute avoidance and restart survival, never latency.
- Fault Tolerance & Delivery Semantics
What is the cheapest thing that can be safely done twice?
What is new here: redundancy is the only mechanism in this whole subject that cannot be amortized.
Read it inRedundancy cannot be amortizedWitness and tiebreaker replicas
- Scheduling & Workload Management
Which work runs now, which work waits, and who pays for the waiting?
Compare hardware in $ per unit of work, not $ per hour.
Read it inCost-aware placement and spot
- Cost Mechanics
What is metered, and what is the smallest increment you are charged for?
The order is stable: the cheapest lever here is organizational and the most expensive is architectural.
Read it inTarget file sizeWhat priced design forbidsScale-to-zero: a one-line break-evenThe four things a vendor can meterThe exchange rate between all the other unitsNetwork: honest shape, strategic sizeStorage: an integral over timeWhich verdicts survive a price change
- Data Modeling
What does one row mean, who is allowed to change that answer, and what breaks downstream when they do?
It is economic contingency, not a permanent law.
Read it inGrain: one row equals one what?
- Observability & Operations
What is the smallest description of a running system from which we can find a fault, fast enough to matter, at a cost we can still pay while the system is failing?
For finding a fault, the signal that matters is a rate, not a total.
Read it inCost and capacity as signals
Related
Made on the same pages most often.