Skip to content
The Data EngineeringThe Data EngineeringBeta
About
Main content

What data quality can claim

What is the single claim of data quality, and what is still unsolved?

The single claim of this mechanism

It is approximated in two ways:

  • against expectations: cheap, narrow, and they go out of date;
  • against second opinions: broad, expensive, and only as good as their independence.

Every mechanism here is a point on one of these two ladders. The two ladders have complementary blind spots, which is why no organisation gets away with only one.

What is unresolved, honestly

  • Partial-failure semantics has no settled answer. Whether a run with 0.01% quarantined "succeeded" is still decided case by case. The completeness claim of a published dataset is almost never expressed in a form a can read.
  • Correction propagation is worse. Restate says what a correction needs, but nobody has a general mechanism for reaching every consumer who already read the wrong value, and the field has not converged on when a fix must be published rather than silent.