Tagging and attribution

How fine can cost attribution be, and how does chargeback fail?

A global fact such as "which team, pipeline or tenant caused this consumption" cannot be reconstructed afterwards, hop by hop. It must travel in-band with the work, from the point of consumption through every aggregation boundary to the bill. This is the same idea as trace context in distributed tracing (an identifier passed along with each request). Tagging is trace context for money. The obstacle is organizational, for the same reason.

In practice it looks like a small, consistent set of labels on every resource:

# Terraform-style example: every resource carries the owner and the pipeline.
tags = {
  team     = "analytics"
  pipeline = "nightly-shuffle"
  env      = "prod"
}

The hard limit

If ten teams share one warehouse and the engine emits no per-query accounting, no tagging scheme can recover who spent what. The information was destroyed at the meter. Resource pools, per-team warehouses and separate credentials are therefore not only isolation mechanisms. They are the measurement apparatus. The sharing loss they cause (less efficient use of the machines) is the price of being able to see anything at all. That is a direct trade between the efficiency you get from sharing and the ability to run the outer loop.

Some costs cannot be attributed

The storage layer, the catalog and the platform team are unattributable in principle. They must be allocated by a rule that is arbitrary by construction. The arbitrariness is not a flaw to be engineered away. It is where the organizational conflict lives, which is why allocation methods consume more meeting time than the spend deserves.

How chargeback fails

Chargeback works, and the way it fails is instructive. Making a team's spend real causes them to optimize their line item, not the total. The classic response is to stop using the shared table and keep a private copy, which lowers their number and raises the organization's. The feedback loop you installed to reduce cost has produced duplication, drift and a correctness liability. Efficiency in the accounting produced fragmentation in the architecture. An incentive is a mechanism, and it has the same failure modes as any other.