Master Data Doesn't Transform Anything: A Playbook for Serving Somebody Else's Transformation (original)

I was asked for a way to align master data management with digital transformation and customer experience. Half of that request rests on a premise I don't accept, and the half I don't accept is the useful half.

The brief on my desk this week asks for a playbook for aligning a master data management program with digital transformation and customer experience goals. It is a reasonable request, it is the one I get most often, and the word doing the damage is aligning — because aligning is what you do with two things of the same kind. Two transformations. Two peers.

They are not the same kind of thing. There is nothing transformative about master data management. Done properly it is an enabler: it produces clean, consistent, de-duplicated data the systems around it can trust, and then it stops, because it will not by itself do anything with that data. Everything of value that happens next happens in somebody else's system, under somebody else's name on the slide.

I am not being modest on the discipline's behalf. A program that believes its own billing takes the lead position in a transformation, and then finds itself accountable for an outcome it is structurally incapable of producing on its own.

So this is a playbook for the honest version: master data standing beside the initiatives that will actually change something, deliberately coupled to them, multiplying what they deliver. The two Mondays before this one covered how that work gets funded and how the executive support behind it survives the first year; I am not re-running either. This is what you do once the money is there and the program has to live inside a machine it does not control.

One warning about what follows. Where I have a procedure you get the procedure, in steps. Where what I have is a judgement call, I will say the word judgement and not dress it up in numbered steps. Exactly one section below carries a real procedure, and that is the honest shape of this work rather than a failure of the article.

What the discipline actually produces, and where it stops

Start with the shape of the evidence on transformations generally, because it is unusually specific about where a data platform sits.

The Boston Consulting Group's 2020 study of digital transformation — its own work with 70 companies plus a survey of 825 senior executives — found that 70 percent of transformations fall short of their objectives: 30 percent met or exceeded target value with sustainable change, 44 percent created some value but missed their targets, and 26 percent produced little or nothing [1]. That headline gets quoted constantly. The part that matters here almost never does.

BCG isolates six success factors and reports that companies which addressed only three or four of them failed, regardless of effort or spend [1]. One of the six is a business-led, modular technology and data platform. So the data foundation is necessary. It is also, on this evidence, one-sixth of the answer, and the other five-sixths are strategy, leadership through middle management, talent, governance and outcome monitoring — none of which a master data program can supply and most of which it cannot influence.

BCG puts the consequence more bluntly than I usually can get away with. Describing how successful transformations scale, they say the winning pattern is to pick one or two lighthouse use cases, build a minimum viable solution, iterate until it works in the market, and then work backward to the platforms and infrastructure required to support it — and that this is radically different from assuming that if you build the platform, the businesses will figure out how to get value from it [1]. That sentence is a description of my discipline's most common failure mode, written by people who were not thinking about my discipline at all.

Among BCG's illustrations of the data-platform factor done well, incidentally, is a company that waited until the late stages of its transformation to upgrade its data infrastructure and architecture [1]. Presented as a positive example. Not a cautionary one.

Now, the fair objection: read the marketing and you will be told master data transforms the business. Read it closely and you will find it does not say that. A representative vendor page describes its platform as transforming fragmented information into a single source of truth, enabling organizations to make reliable decisions [5]. Look at the object of that verb. The thing being transformed is the information. What follows — the decisions, the engagement, the revenue — arrives under enabling, which is a claim about a precondition, not about an outcome. The copy is more careful than its readers are.

I went looking for the strongest case against this before writing it down, and I found one worth conceding. Richard Churchill of Leading Resolutions lists, among three predictable sequencing failures, launching digital platforms, analytics or operating-model redesign while core processes, data and governance remain fragile — The ambition is right, but the sequence is wrong — and offers a financial services firm that built its foundational data structures before introducing new applications, reporting faster delivery and higher adoption for it [10]. Foundation-first, and it worked.

So the claim needs to be more exact than I first made it, and this is the version I will hold to for the rest of the piece: leading and going first are different things. Do the data work early if the dependencies demand it. What a master data program should not do is take the transformation's name and accept the transformation's outcome metrics as its own success criteria. Churchill's second failure mode is the reason — stabilization work that runs without visible strategic progress attached drains executive confidence [10]. Notice that BCG's late-upgrade example and Churchill's early one point opposite ways on timing, which is itself the finding: the evidence does not settle the schedule, so the schedule is not where the claim lives. Going first is a schedule. Leading is an accountability, and it is the one that cannot be met.

That is a stance about positioning, not a procedure, and I am not going to give you a decision tree that manufactures it out of your circumstances. What I can do is make the consequences concrete, which is the rest of this piece.

Interdependence is the ticket and the tax

The moment you stand beside a larger program, you have bought interdependence, and you have bought all of it, in both directions.

The project portfolio literature has a decent vocabulary for it. Projects are interdependent when the success of one depends on others, and the standard taxonomy names resource interactions, benefit interactions and technical dependence, with outcome and learning dependencies layered on top [6]. Master data programs generate all five at once, which is why they are hard to schedule and harder to defend.

What you get in exchange is real. A combined initiative is treated as one unit by the people funding it. That unit gets a name, an executive, a line in the plan, and a budget that a standalone data-hygiene program of the same size would never have been given. For a great many master data programs, riding inside a modernization or a migration is not the clever option. It is the only door.

What you pay is that the unit succeeds or fails together. Somebody else's missed deadline, somebody else's thin deliverable, somebody else's turnover — all of it lands on the program, and none of it is anything you did.

The decision about whether to couple is a judgement. I want to be plain about that, because the temptation here is to invent a scoring model, and a scoring model would be a confidence trick. What is not a judgement is knowing what you are signing, and there the literature supplies something you can do. Build the dependency map before you commit, not as a status artifact afterwards. Two moves earn their keep:

• Re-draw the portfolio sized by dependency rather than by investment. Killen and colleagues show the same portfolio in two views — projects sized by money spent, and projects sized by the number of other projects depending on them — and the two pictures identify different critical projects [6]. Investment is the proxy everyone reaches for. It is not the same question. Find out which view you are on: a small master data workstream carrying six dependents is a different risk object from a large one carrying none, and only one of those two is going to be defended when the initiative is cut back.
• Walk the chain two hops, not one. A dependency matrix records that A depends on B and separately that B depends on C, and the resulting dependency of A on C is not visible on it; the paper's worked example has a project whose stated dependencies number two while the multi-level chain shows its success depends on four additional projects [6]. That is the arithmetic behind most surprises. Nobody hid anything. The instrument does not show second-order relationships, so nobody looked.

That gives you the shape of what you are joining. It does not make the call. Making the call is your job and it stays your job.

The move that makes it visible: put master data in the type-ahead

Coupling to a customer experience program raises a harder problem than funding. The master data deliverable is invisible. Its output is an absence — the duplicate that was not created, the address that did not bounce, the report that stopped disagreeing with the other report. Absences do not survive a steering committee, because there is no demonstration to give.

The pattern I would build first, every time, is lookup-before-create with in-flight cleansing, wired into the interface the program's users actually touch. Almost every modern interface uses type-ahead. Connecting the master data system to that field through a service interface — a search call as the user types, returning candidate existing records with their identifiers, plus standardized values for the fields being entered — puts master data directly into the user experience rather than downstream of it.

It produces visible value on the customer experience program's own terms. The user sees a suggestion appear, picks the existing customer instead of typing a new one, and finishes faster. That is a demonstrable interaction, and it belongs to the program that owns the interface.

It fits what the analysts recommend when they are careful. Gartner's advice at its 2023 marketing symposium, from the same sessions that told people to stop chasing complete customer views, was to identify narrow customer data use cases and start with low-hanging fruit — think big but start small and move and iterate and learn fast [4]. That is BCG's lighthouse pattern, arriving from a different firm at a different conference [1].

And it moves entity resolution to the one place where it is cheapest and most accurate: the moment of capture, in front of the person who has the most context. The user filling in that form is looking at the customer, or talking to them. Any question the match logic cannot settle, that person can settle, at no marginal cost and with no queue. Hold on to that, because it is the hinge of the next two sections.

What this is not is a step-by-step procedure, and I will not pretend otherwise. It is a named pattern with real preconditions — a service interface on the hub, a latency budget the interface owner will accept, and a negotiation with a team whose backlog you do not control. Whether those preconditions are met is site-specific. The pattern is not.

Customer 360: two promises, and both of them are unbounded

Which brings us to what gets sold. The customer 360 pitch makes two promises: comprehensiveness out of the box, and automation. Both are moving targets, and they move for different reasons.

Comprehensiveness cannot be delivered out of the box because comprehensive has not been defined. That is not a rhetorical point; the vendors say it themselves. One platform's own explainer opens by conceding that the exact definition varies depending on who you ask — a central hub to some, the output of a master data or warehouse layer to others, and to some vendors something that requires enrichment with third-party and social data [5]. A product cannot arrive complete against a specification that three vendors define three ways.

Gartner's people went considerably further at their 2023 marketing symposium than I would have expected. Ben Bloom, presenting research he himself flagged as contrarian, told the room to give up on the 360-degree view of the customer because the cost of chasing it becomes unmanageable, citing an earlier finding that only 14 percent of organizations reported having achieved one; his colleague Matt Wakeman reported that 45 percent of marketing teams using a customer data platform for both collection and activation still had data integration problems, and that teams with the most integrations had about as much trouble proving marketing's value as teams with none [4].

The mechanism Bloom gives is the one worth carrying: integration cost grows with the pairs, not the sources. Four channels need six cross-channel integrations; twenty need on the order of two hundred [4]. Check the arithmetic yourself before repeating it — pairwise combinations of twenty is 190, so the figure as reported is rounded — but the shape is right and the shape is what bites. Adding the eleventh source is not one more integration. It is ten more.

This position is contested on the record. In the same report, Redpoint Global's John Nash argues you do not have to give up the golden record because you need it to create the moments that drive revenue, and LiveRamp's Jessica Shapiro says flatly that without the 360-degree foundation she does not know how you would find the right customer at the right point in their journey [4]. Both work for firms whose products presuppose the thing they are defending, and both are making an argument that stands anyway. Nash gives ground where it counts: he says explicitly that you should not collect data indiscriminately, and should start with the more valuable data linked to business objectives and use cases [4]. Strip the vocabulary and Gartner's contrarian and the CDP vendor's strategy chief recommend the same first move.

So here is the doable version, and it is a discipline rather than a procedure. Before anything is purchased, write the definition down: the attribute list that is in scope, the source systems that are in scope, a named owner for each attribute, and — the part that gets skipped — an explicit out-of-scope list naming what this customer 360 will not contain. Then the comprehensiveness promise has something to be measured against, and the answer to "is it complete?" becomes checkable instead of aspirational. Nobody can sell you completeness against a boundary you wrote.

The second promise, automation, is where I can finally give you a procedure.

Thresholds: the one place I can hand you a real procedure

Master data cannot be fully automated. There are two ways to appear to have automated it anyway. The first is to force every record into a fixed structure so that nothing can be ambiguous, which trades a matching problem for a modeling one. The second is to set the auto-approve threshold low enough that almost everything merges by itself. The second is more common, because it looks like tuning.

It is worth being precise about what that dial is, because the formal answer is old, settled, and rarely quoted to the people turning it. Probabilistic record linkage as the field practices it descends from Fellegi and Sunter's 1969 framework, which considers three actions for any pair of records — link, possibly link, or do not link — and controls two error probabilities: the probability of linking records that do not match, and the probability of failing to link records that do. Two thresholds fall out of those two error rates. Above the upper one you link; below the lower one you do not; between them the pair is a possible link and goes to clerical review. Their result is that this procedure is optimal, and specifically that no other rule produces a smaller clerical review region while holding the error rates at that level [2][3].

Two structural facts make the conversion worse than a proportional trade.

The first is transitivity. In most master data settings, knowing that A matches B and B matches C means A matches C, and enforcing that introduces dependencies between pairs that the pairwise model does not represent [2]. So a threshold set slightly too low does not produce a scattering of extra bad pairs. It produces chains, and the chains close into clusters. One over-eager rule on a common surname and a shared postal code can weld a family, a company's branch offices, or a dozen unrelated people into a single golden record.

The second is that the undo is structural, not clerical. Vendor documentation is the right place to see this, because it describes the mechanism rather than the marketing. An automerge rule combines two or more records automatically, without manual intervention [7]. The result is a tree: merge a1 and a2 into a, merge b1 and b2 into b, then merge a and b into c, and unmerging a is an operation on that tree that pulls a sub-tree out of the structure — the outcome depends on the order things were merged in, not only on which record was wrong [8]. And an unmerge is not one operation but two choices: unmerge the cross-reference record alone and the child records stay attached to the record they were wrongly merged into, or unmerge with lineage and the children return to where they were before [9]. Somebody has to make that call, per record, after the business has been transacting against the merged version for months.

Which is the whole problem in one sentence: the cost does not land where the shortcut was taken. The person who lowered the threshold saw a queue shrink. The stewards inherit the remapping, and they inherit it after reports have been issued and decisions made on the merged records.

The procedure

Six steps. Every one of them is checkable by someone who was not in the room.

1. Build a labeled sample before you touch a threshold. A few hundred pairs, adjudicated by somebody who knows the domain, drawn deliberately from the hard region — near-duplicates, shared addresses, name variants — rather than sampled at random, because a random sample of all possible pairs is almost entirely obvious non-matches and will tell you nothing. Carry the caveat the researchers carry: labeling errors and the sampling procedure itself introduce bias into the estimates, and correcting for that is an open research problem, not a solved one [2]. A biased ruler still beats no ruler, as long as you say which one you used.
2. Declare the two error rates first, as a business decision, and derive the thresholds from them. That is the direction the theory runs and it is almost always run backwards in practice: somebody picks a target automation percentage and backs into the thresholds that produce it. Ask the business owner what rate of wrongly merged customers is acceptable, in words, and write the answer down. If nobody will answer, that is the finding — you have discovered that no one is willing to own the error, which is a governance problem and not a tuning one.
3. Price the two errors separately and never trade them at par. A false split costs you a duplicate: irritating, visible, cheap to fix. A false merge produces a customer record containing another person's data — their orders, their address, possibly their consent flags — and depending on your jurisdiction and domain that is a privacy incident rather than a data quality defect. Any tuning conversation that treats the two as symmetric errors on one dial has already gone wrong.
4. Demonstrate the unmerge before go-live, on a chained merge, and record which semantics you get. Not a two-record merge; a three-or-more-record chain with child records attached, so you see the tree behavior and the lineage choice [8][9]. If nobody can show you the undo working on that shape, you do not have an auto-merge threshold. You have a one-way door with a threshold painted on it.
5. Instrument the review queue as a rate, not a backlog. Arrivals per day against dispositions per day. A queue with a thousand items in it is not a problem statement. A queue whose arrival rate has exceeded its disposition rate for three weeks is, and it is a staffing or a rule-quality problem long before it is a threshold problem.
6. Never move a threshold to clear a backlog. This is the cheat, stated as the rule that forbids it. If the queue is too big, the honest levers are better match rules, better source data, or more steward capacity. Lowering the bar does not process the queue; it deletes the queue and keeps the work, moved somewhere nobody is counting it.

One note on the present moment, because it changes the arithmetic rather than the method. The pressure to raise automation rates is arriving on nearly every program right now with artificial intelligence attached to it, and the downstream consumer of a golden record is increasingly a model rather than a report. That matters for step three. A false merge feeding a report is a wrong number somebody may eventually query, because reports have readers who know their own business. A false merge feeding a retrieval index or a training set is a wrong number nobody queries, because nothing in that path is designed to be surprised. The error rate you accept at the merge is inherited silently by everything built on it — an argument for tightening the threshold in exactly the season everyone is being asked to loosen it.

What I would do first

Go and read your own charter, and strike any success criterion you cannot move on your own — that is the line between going first, which is a schedule you argue on the merits, and leading, which is an accountability nobody can meet. Write every deliverable that survives as a multiplier on somebody else's. Build the type-ahead integration before the warehouse feed, because one of them can be watched and the other cannot. Before you evaluate a customer 360 product, write down what it will not contain. And when you reach the matching rules, declare your two error rates before you touch a threshold, price a false merge as a different kind of event from a false split, and make somebody demonstrate the unmerge on a chained merge while you watch.

None of that transforms anything. It makes the transformations around it work, which is a smaller sentence and a better job.

I keep bees, and the part that maps is not the honey. Nothing the hive produces is made by the beekeeper. What the beekeeper does is keep the colony in a condition where the work the bees are already doing does not fail — and the season a keeper decides the hive is the point rather than the orchard is the season it all goes quietly wrong.

Tomorrow, Isabel takes the same territory into a platform and asks what a product can and cannot absorb of the matching problem. Standing disclosure, unchanged: the practice behind this publication is a certified Profisee implementation partner, paid by clients for deployment work rather than by the vendor for the sale, and our services revenue is downstream of this category of software being chosen at all. Nothing in this piece argues for buying anything, and the vendor documentation cited here is cited for how a mechanism behaves, not as a recommendation or a criticism of any product. Weigh her criticisms above her praise regardless.

Master data isn't a project you finish. It's a hive you keep — and the hive is not what you are there for.