Opacity forfeits optimization
Opacity forfeits optimization.
What it is
Opacity forfeits optimization. Opacity buys generality and gives up optimization, and the forfeited work reappears at a higher layer.
From Storage: Physical Organization › Values, documents, order
Some optimizers perform this inversion for a few known functions. Most do not, for a sound reason: inverting an arbitrary function is generally impossible. The consequence is stark: the simplifier's coverage decides which predicates reach the storage layer at all. Pruning, skipping and pushdown all come after the question "did the predicate survive normalization in a matchable form?" A wrapped column is not a small inefficiency. It silently disables the entire data-reduction machinery for that query.
From Query Planning & Optimization › Folding and sargability
The consequence is severe: your bill becomes a function of physical layout and not of the question you asked. The same query costs different amounts on different days, as the table changes. Two queries with identical meaning differ in price because of the compaction state (how well small files have been merged) of the files underneath.
From Cost Mechanics › Scan pricing
Where it appears
In 3 mechanisms. Each says what the mechanism is about, then how this principle shows up there, in its own words.
- Query Planning & Optimization
How does a system choose a plan for a question when the facts that decide the best plan are unknown until the plan has run?
It silently disables the entire data-reduction machinery for that query.
Read it inPredicate simplification, constant folding and sargabilitySubquery decorrelation
- Cost Mechanics
What is metered, and what is the smallest increment you are charged for?
Pricing is where physical data independence leaks, and it leaks in dollars, not in errors.
Read it inIdle warehouses and SELECT *Abstractions succeeding$/TB scanned: a proxy for compute
- Schema Management & Evolution
What happens when the shape of your data changes on its own schedule, and nobody can change the readers, the writers and the old data at the same moment?
Semantic types. currency_code instead of string. price_in_cents as int64 instead of plain int64.