Skip to content
About
Main content

Conformed dimensions, the bus matrix, role-playing dimensions and reference data

How do many teams make their numbers comparable without N² private translations?

Why conformance is a coordination problem

This is where arrives as a first-class quantity.

Two can be compared along a only if that dimension means the same thing in both: the same key domain (the same set of identifiers), the same attribute definitions, and the same member population (the same list of customers, the same list of products). That agreement is called conformance, and a is its physical result: one shared dim_customer that every fact table points to.

The economics

The economics are the strongest argument in this mechanism, so we write them out.

Suppose Fernwood has N fact tables (one per business process: orders, shipments, returns, support tickets, web sessions...). Suppose no dimensions are shared.

  • To answer any question that crosses two processes, someone must build a mapping between the two processes' versions of an entity (for example, "customer" in orders and "customer" in tickets).
  • Over all pairs of tables that is N(N−1)/2 reconciliations, which is O(N²). Each is built by hand, each is re-derived by whoever asks.

With conformance, each fact table agrees once with the shared dimension. That is N agreements, which is O(N).

N (fact tables)Pairwise reconciliations N(N−1)/2Agreements with a shared dimension (N)
5105
8288
2019020
501,22550
teams, systems or record sources that must agree
pairwiseshared vocabulary
Data table
Agreements to maintain, by parties
Parties NseriesAgreements
2pairwise1
2.1pairwise1.16
2.21pairwise1.33
2.32pairwise1.52
2.43pairwise1.74
2.55pairwise1.98
2.68pairwise2.26
2.82pairwise2.56
2.96pairwise2.89
3.11pairwise3.27
3.26pairwise3.69
3.42pairwise4.15
3.6pairwise4.67
3.78pairwise5.24
3.97pairwise5.88
4.16pairwise6.59
4.37pairwise7.38
4.59pairwise8.25
4.82pairwise9.22
5.06pairwise10.3
5.32pairwise11.5
5.58pairwise12.8
5.86pairwise14.3
6.16pairwise15.9
6.47pairwise17.7
6.79pairwise19.7
7.13pairwise21.9
7.49pairwise24.3
7.86pairwise27
8.26pairwise30
8.67pairwise33.3
9.11pairwise36.9
9.56pairwise40.9
10pairwise45.4
10.5pairwise50.3
11.1pairwise55.8
11.6pairwise61.8
12.2pairwise68.5
12.8pairwise75.8
13.5pairwise83.9
14.1pairwise92.9
14.9pairwise103
15.6pairwise114
16.4pairwise126
17.2pairwise139
18.1pairwise154
19pairwise170
19.9pairwise188
20.9pairwise208
22pairwise230
23.1pairwise254
24.2pairwise281
25.4pairwise311
26.7pairwise343
28pairwise379
29.4pairwise419
30.9pairwise463
32.5pairwise511
34.1pairwise564
35.8pairwise623
37.6pairwise688
39.5pairwise760
41.5pairwise839
43.5pairwise926
45.7pairwise1.02K
48pairwise1.13K
50.4pairwise1.25K
53pairwise1.38K
55.6pairwise1.52K
58.4pairwise1.68K
61.3pairwise1.85K
64.4pairwise2.04K
67.6pairwise2.25K
71pairwise2.49K
74.6pairwise2.74K
78.3pairwise3.03K
82.2pairwise3.34K
86.4pairwise3.69K
90.7pairwise4.07K
95.2pairwise4.49K
100pairwise4.95K
2shared vocabulary2
2.1shared vocabulary2.1
2.21shared vocabulary2.21
2.32shared vocabulary2.32
2.43shared vocabulary2.43
2.55shared vocabulary2.55
2.68shared vocabulary2.68
2.82shared vocabulary2.82
2.96shared vocabulary2.96
3.11shared vocabulary3.11
3.26shared vocabulary3.26
3.42shared vocabulary3.42
3.6shared vocabulary3.6
3.78shared vocabulary3.78
3.97shared vocabulary3.97
4.16shared vocabulary4.16
4.37shared vocabulary4.37
4.59shared vocabulary4.59
4.82shared vocabulary4.82
5.06shared vocabulary5.06
5.32shared vocabulary5.32
5.58shared vocabulary5.58
5.86shared vocabulary5.86
6.16shared vocabulary6.16
6.47shared vocabulary6.47
6.79shared vocabulary6.79
7.13shared vocabulary7.13
7.49shared vocabulary7.49
7.86shared vocabulary7.86
8.26shared vocabulary8.26
8.67shared vocabulary8.67
9.11shared vocabulary9.11
9.56shared vocabulary9.56
10shared vocabulary10
10.5shared vocabulary10.5
11.1shared vocabulary11.1
11.6shared vocabulary11.6
12.2shared vocabulary12.2
12.8shared vocabulary12.8
13.5shared vocabulary13.5
14.1shared vocabulary14.1
14.9shared vocabulary14.9
15.6shared vocabulary15.6
16.4shared vocabulary16.4
17.2shared vocabulary17.2
18.1shared vocabulary18.1
19shared vocabulary19
19.9shared vocabulary19.9
20.9shared vocabulary20.9
22shared vocabulary22
23.1shared vocabulary23.1
24.2shared vocabulary24.2
25.4shared vocabulary25.4
26.7shared vocabulary26.7
28shared vocabulary28
29.4shared vocabulary29.4
30.9shared vocabulary30.9
32.5shared vocabulary32.5
34.1shared vocabulary34.1
35.8shared vocabulary35.8
37.6shared vocabulary37.6
39.5shared vocabulary39.5
41.5shared vocabulary41.5
43.5shared vocabulary43.5
45.7shared vocabulary45.7
48shared vocabulary48
50.4shared vocabulary50.4
53shared vocabulary53
55.6shared vocabulary55.6
58.4shared vocabulary58.4
61.3shared vocabulary61.3
64.4shared vocabulary64.4
67.6shared vocabulary67.6
71shared vocabulary71
74.6shared vocabulary74.6
78.3shared vocabulary78.3
82.2shared vocabulary82.2
86.4shared vocabulary86.4
90.7shared vocabulary90.7
95.2shared vocabulary95.2
100shared vocabulary100

This has the same structure as the argument for a shared interchange data format (such as Apache Arrow, a common in-memory column format) versus writing a converter for every pair of systems. It is the same law:

The bus matrix

A lists business processes down the side and dimensions across the top. A mark means "this process's fact table uses this dimension".

Process (fact table)DateProductCustomerWarehouseRegion (via customer)
OrdersXXXXX
ShipmentsXXXXX
ReturnsXXXXX
Support ticketsXXX
Web sessionsXXX

The bus matrix is not documentation. It is a map of which questions are answerable at all. Each filled cell is a negotiated agreement that someone paid for. An empty cell is a question that will cost a project. In the table above, "support tickets per product" is impossible today, because the tickets table has no product dimension.

Role-playing dimensions

A is one physical table used in several roles. The calendar is the classic case: an order has an order date, a ship date and an invoice date. There is one dim_date table, but the fact has three to it.

The safe way to expose it is three named views, so the column names carry the role:

CREATE VIEW dim_order_date   AS SELECT date_key AS order_date_key,   date AS order_date,   month AS order_month   FROM dim_date;
CREATE VIEW dim_ship_date    AS SELECT date_key AS ship_date_key,    date AS ship_date,    month AS ship_month    FROM dim_date;
CREATE VIEW dim_invoice_date AS SELECT date_key AS invoice_date_key, date AS invoice_date, month AS invoice_month FROM dim_date;

It is cheap to build. The failure mode is a consumer joining on the wrong role and getting a number that is plausible and wrong ("revenue in March" by ship date is not "revenue in March" by invoice date). Naming is the only defence.

Reference data

Reference data means currency codes, country lists, product taxonomies, status codes (also called controlled vocabularies). These are the smallest tables in the platform and have the highest fork rate, exactly because they are small enough to copy instead of asking the owner. Every fork is a silent divergence. Months later it appears as two reports disagreeing by what everybody calls a rounding error, which is not a rounding error. (For example: two currency tables, one with the rate "1.0845" and one with "1.08450001", produce two revenue totals that differ by a few dollars. Everyone blames rounding. The real cause is two sources for one fact.)

A note on decentralization

A or mesh arrangement (each domain team owns and publishes its own data, with no central warehouse team) does not remove the O(N²) term. It relocates it into cross-domain and a shared identifier scheme. That is the same agreement with a different owner. This is the cost-conservation law again: , and any architecture that claims to eliminate it has hidden it.