Asset identity: what is the row keyed on?

Why does keying metadata on a name turn every rename into a silent delete?

CaseConstructed case · Hana at Trailmix

A rename, and the table's history vanished

Hana renames the table orders to sales_orders to match a naming standard. Within a week three odd things happen.

The lineage graph for sales_orders shows no upstream tables and no downstream dashboards. The history of who read the table is empty.
A cleanup bot, which looks for tables "with zero usage in 90 days", proposes to delete sales_orders.
Separately, a query on the events table (hourly partitions, three years old, many tiny files) spends 40 seconds in planning before it reads a single byte of data. Meanwhile a streaming job that writes to the same table every few seconds has begun to fail with commit errors.

Nothing is corrupt. We need to see how a catalog names things, how the metadata store scales, and how writers commit.

Everything here is a join across systems, and the join key is the asset identifier. So the first question is not "what metadata should we store?" but "what is each record keyed on?"

Suppose identity is a path or a name, such as catalog.schema.table, or worse a folder prefix. Then a rename is a delete plus a create. Every edge in every graph keyed on the old name silently breaks. Nothing raises an error:

the lineage graph becomes two disconnected pieces;
the ownership record points at a name that no longer exists;
the usage history restarts at zero, which makes the table look unused, and therefore deletable.

That is exactly Hana's problem 1 and 2. The cause is informational: a human-meaningful name cannot also be a stable key, because the two properties conflict. Names change because meaning changes.

The fix is always the same: give each asset a meaningless stable ID at creation, treat the name as an ordinary attribute, and resolve names to IDs at access time. (This is called late binding: the name is looked up when used, not fixed in advance.)

You may know this mechanism already:

Granularity	Name-based approach	ID-based approach
Row	natural key (email address)	surrogate key (an auto-number)
Column	find a column by its name in the file	find it by a column ID stored in the table metadata
Table	`catalog.schema.table`	asset UUID

That is three appearances of one mechanism: stable meaningless identity, late-bound names.

asset_id  : 7f3a-91c2-...         (never changes)
name      : sales_orders          (changed from "orders" last week; just an attribute)
owner     : team:checkout-data    (resolved at query time)
lineage edges, usage history, tags all reference asset_id, never the name

The cost is the usual one: an indirection layer that nobody can read by eye, plus a resolution service (name to ID) that is itself a catalog with its own availability requirement.

Cross-system identity remains unsolved. The same physical table has one identity in the metastore, another in the job scheduler, another in the BI tool and another in the lineage store. The join between them is a naming convention (often a URN, a standard text form for an identifier), and conventions are asserted, not enforced. That is why every metadata interchange standard spends most of its text on naming and still does not fully fix it.

Identity is the join key of the whole topic. If identity is a path, every rename is a silent delete.