The Same Fight

Weaving Intelligence (House AI Style), with Jeff Shabel’s Experiential Commentary

The tools for the next phase of governance already exist, and they share one idea: more of the work of enforcing a policy moves out of the hands of the people who own the data and into the systems that carry it. Rules will sit in the pipeline. A data contract will refuse a load that breaks it. A tag will switch on a masking policy without anyone signing a form. And a model will read each record against the policy and flag, or quietly correct, the ones that do not fit. The question put to me was what happens to adoption when the system refuses the non-compliant record. Does it stop being a people problem, or does the fight simply move upstream, to whoever writes the rule and whoever can grant the exception? I gave neither answer:

It’s the same fight: who has responsibility for changing the record and who decides to what it should be changed. Whether it’s the system that’s making the changes or the Steward, Ownership and Responsibility are the terrain on which the battle is being fought.

Those are my words; where the prose goes past them and the sources, it says "my reading".

That answer is a forecast, and I want to be plain about what kind. Rules engines are not new. Building them, and the metadata-driven processing around them, was my own niche for more than a decade, and on their own promise of efficiency and time to market they mostly delivered; where they struggled was adoption and fitting into people's daily work. What nobody has yet watched run for a decade is enforcement in which a model reads every record against policy. What I have is the pattern of how governance policies get adopted or ignored today, and a view of which parts of that pattern a system can change. The claim is that automation changes where the argument is held and how fast it is settled, and does not change what the argument is about. Two questions decide whether a policy is adopted: who is responsible for changing a record that breaks it, and who decides what the record should be changed to. A steward with a queue and a pipeline with a rule are both answers to the first question, and neither answers the second.

Enforcement is already moving into the systems

The direction is not speculative; the tools for it exist and are documented. Zhamak Dehghani's data mesh names the model "federated computational governance": a decision-making model "led by the federation of domain data product owners and data platform product owners," with "automated execution of decisions by the platform" [2]. The Open Data Contract Standard, now at version 3.2.0 under the Linux Foundation, gives a data producer a machine-readable place to declare quality rules across the familiar dimensions, accuracy, completeness, conformity, consistency, coverage, timeliness and uniqueness, so a pipeline can check a delivery against its contract [3]. On the access side, Databricks documents attribute-based access control in Unity Catalog in exactly the terms the forecast uses: "When a securable object has the attributes targeted by a policy, that policy takes effect automatically," so "a single policy can enforce consistent access rules across an entire metastore, catalog, or schema" [5].

The next step is the one the question named: a model reading records against policy rather than a fixed rule reading fields. Profisee's Malcolm Hawker described the shift as one "from human-first (where people define the rules) to human-led (where people review and approve the rules suggested by AI)" [9]. Hawker writes for Profisee, the vendor our practice partners with and refers clients to, so weigh his account of where the field is going with that interest in mind; I cite it because it states the direction clearly, not because it is neutral.

So the system will increasingly do the refusing. The hope behind the question is that refusal ends the argument: the record does not get in, so there is nothing to adopt or ignore. The worry behind it is that the argument does not end but relocates, and that the people who used to argue about the record will now argue about the rule. My answer cuts across both, and the rest of this piece is why.

What a rule actually is

A rule in a pipeline is a decision somebody made, written down in a form a machine can apply without asking anyone. That is its whole value and its whole limit. It applies the decision faster and more consistently than a person would, and it applies exactly the decision that was written, including when the decision was wrong or has gone stale.

Look at what the contract standard asks for alongside the rule itself. Each quality rule can carry a severity, defined as "The severity of the DQ rule," and a businessImpact, defined as "Consequences of the rule failure" [3]. A machine can evaluate the rule. It cannot fill in either of those fields. Somebody has to decide how bad a failure is and what it costs, and those two judgements decide whether the rule blocks a load, warns about it or is quietly ignored. The tag-driven access policy has the same shape one level down: the policy takes effect automatically on anything carrying the tag [5], which means the decision has moved from the moment of access to the moment somebody applies the tag and writes the policy. Automatic enforcement does not remove the decision. It moves it earlier and makes it harder to see.

Hawker's version moves it again. When the model suggests the rule and a person approves it [9], the deciding is the approval. I said in October that policy enforcement and routing are work that well-written algorithms already do, and that I expect AI to take them over anyway, often because it is good enough rather than better (The Primrose Path). Nothing here changes that. The point for adoption is narrower: whoever writes the rule, a person or a model, someone has to accept it as the enterprise's rule, and accepting it is the same act as deciding what the record should become.

Two questions the engine does not answer

The first question is who is responsible for changing the record. A system that refuses a load has identified a problem; it has not fixed one. The record is still wrong at its source, and the refusal lands on someone. In a programme run by people, that someone is a steward working a queue. My reading is that in a programme run by a pipeline, it will be whoever owns the producing system, if anyone has been named, or whoever is nearest the failure when the downstream report goes blank. I have argued that the correction has to be made where the data is produced, or where a calculated value is derived, and that a fix applied downstream is a patch (Two Models and a Seam). A refusal makes that sharper, not softer. It stops the bad record, and it puts the cost of the bad record on the producer's doorstep immediately instead of a month later. Whether the producer accepts that cost is exactly the adoption question, unchanged.

The second question is who decides what the record should be changed to. That is a question about meaning: which definition of an active customer, which product hierarchy, which tolerance on a price. The rule encodes one answer. If the answer is contested, encoding it does not settle the contest; it settles who wins it, in favour of whoever controls the rule. The rule I have given for contested definitions is that the decider decides, and that the decider is the person who holds both the authority and, more importantly, the power to enforce the decision (Rules of Decreasing Strictness). My reading is that automation does something specific to that rule: it hands the decider a perfect enforcer. Today enforcement power is a matter of sponsorship and escalation, of who can make another department's director do something. In a pipeline it is a matter of who can change the configuration. That makes the fight over who decides more consequential, because the winner no longer has to chase anyone. The rule does it for them.

Asked earlier how a missing ownership map shapes a programme, I described a pattern rather than a case: where nobody holds the decision, a group steps in to fill the gap, and the outcome turns on whether the level of the organisation able to confirm their authority actually does so. My reading is that a rule engine with no named owner is that vacuum in its purest form. Somebody has administrator rights. If nobody has been given the decision, the person with the rights holds it by default, and the departments the rule refuses will treat it the way they treat any authority nobody senior has confirmed. That is where the old failures come back: shadow copies of the data that skip the pipeline, manual loads through a side door, a spreadsheet that holds the version of the hierarchy the sales team actually uses.

I have argued before that every governance programme, councils and stewardship models and policy-first approaches alike, lives or dies on its champion, because each asks people to spend time on something that does not touch their own work, and without real weight from the executive floor they give it lip service (Rules of Decreasing Strictness). My reading is that this is the part of the pattern a system changes least. A rule can refuse a record. It cannot make the producing team fix the source, and it cannot make the business accept the definition it enforces. Those still need someone with standing behind them.

Where the fight goes: the exception

If the rule is the decision, the exception is where the decision gets argued after the fact. Every enforcement system that works at scale has had to build an exception route, and the mature ones treat it as a governance object in its own right. Kyverno, a CNCF policy engine for Kubernetes, is a useful example from outside data management because its documentation is explicit about the structure. Its exceptions exist because "a team responsible for policy authoring and administration may not be the same team responsible for submission of resources" [4]. They are switched off until someone turns them on ("PolicyExceptions are disabled by default"), and the documentation's guidance for its older, now-deprecated exception type adds that their use "can and should be controlled by a number of different mechanisms," and that "it is considered a best practice to only allow very narrow exceptions to a much broader rule" [4]. Three separately controlled things, then: who writes the rule, who submits the work, and who may excuse the work from the rule. That is the same fight with the parties written into the configuration.

Clinical software has run this experiment for longer than data management has, and with higher stakes. A hard-stop alert is the medical record system refusing the order. A systematic review by Emily Powers and colleagues found that "Although hard-stop alerts can improve safety, they have been shown to result in unintended consequences," among them "delays in care," and concluded that "end-user involvement is essential to inform design and appropriate workflow integration" [6]. I read the summary of that review, not the full paper, so I will not lean on its details. But my reading is that its conclusion is the forecast stated in another field: the refusal works when the people it refuses had a hand in the rule, and imposes costs when they did not. The law has started to say the same about AI systems that make consequential decisions. Article 14 of the EU AI Act requires that the people overseeing a high-risk system be able "to decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output of the high-risk AI system" [8]. Most master data pipelines are not high-risk systems under that Act, and I cite it only for how the law frames the point: an automated refusal that nobody can override is treated as a defect, not a goal.

I went looking for a study of exception and override rates in data governance rule engines specifically, as opposed to clinical alerts or infrastructure policy. I did not find one, and I will not offer a number. My reading is that the exception queue is where adoption will be measured in an automated programme, because it records every place the rule and the business disagreed and who settled it. A programme whose exception queue is empty is either enforcing rules nobody disputes, or has rules nobody bothers to dispute because they have found a way around the pipeline.

The case that the system ends the argument

There is a serious objection, and part of it is right. For some rules, system enforcement does end the argument. Nobody holds a meeting about whether a date field must contain a date. A mandatory key, a format, a type, a reference to a code list that already exists: once those are checked at the door, the argument about them stops, because there was never a real argument, only neglect. The clinical evidence points the same way. The review found hard stops can improve safety and process outcomes [6]. A refusal changes behaviour, and that is not a small thing.

So the claim needs narrowing, and the narrower version still rules things out. Automatic enforcement ends the argument for rules whose content nobody could reasonably dispute. For any rule that encodes a choice someone could dispute, what a field means, whose definition wins, how severe a failure is, who must fix it, the argument does not end. It moves to the rule's ownership and to the exception route. On that version, an enforced rule that encodes a contested choice, with no named owner and no exception route, should not stay in force without shadow processes growing around it. If programmes routinely ran contested rules that way and the business simply complied, the claim would be wrong.

The best general evidence I could find is consistent with it rather than proof of it. Gartner predicted in 2024 that "By 2027, 80% of data and analytics (D&A) governance initiatives will fail due to a lack of a real or manufactured crisis" [1]. That is an analyst forecast without a stated method, so it shows where the analysts' attention is, not a measurement. The same release quotes its analyst, Saul Judah, saying that "A D&A governance program that does not enable prioritized business outcomes fails" [1]: the failure it describes is one of business outcomes, and my reading is that it is not a failure of tooling. A tool closes a tooling gap, and the tools in the first section of this piece already exist.

The second objection is that the models will soon decide well enough that the question of who decides stops mattering. I take that up in "The same query" below, where the evidence is.

What to learn now

The last part of the question was what a practitioner should learn now that will still matter then. My answer was three things:

Practitioners need to learn why the systems do what they do, how to configure and control them and where the moving line currently is between systematized and manual approvals. This is valuable now, in the past and will continue to be of value in the future.

Each of the three follows from the argument above. The first, why the systems do what they do, is the ability to read a refusal and trace it to the decision behind it: which rule fired, who wrote it, what definition it encodes and when that definition was last agreed. Without that, a practitioner can only escalate or work around, and working around is how the shadow copies start. Lisanne Bainbridge's study of automated control rooms made the same point four decades ago: the designer who tries to remove the operator "still leaves the operator to do the tasks" the designer could not automate, and "the more advanced a control system is, so the more crucial may be the contribution of the human operator" [7]. The tasks left over are the ones that need someone who understands the whole system.

The second, how to configure and control them, is the enforcement power itself. In a programme where rules are configuration, the person who can change the configuration holds the decider's tool. A practitioner who cannot read or change it is a petitioner to whoever can.

The third, where the moving line currently sits between systematised and manual approval, is my phrase from the answer and the most important of the three. My reading is that the line moves one rule at a time, and every move is a decision with an owner: this check no longer needs a person, this approval now does. A practitioner who knows where the line is today can see when it moves, argue about whether it should, and notice when a rule has crossed it without anyone deciding.

I gave a different answer to a similar question in October. Asked what a practitioner should learn once AI takes over enforcement, routing and most exceptions, I said the durable skill was providing value to the business, solving or at least identifying its problems (The Primrose Path). The two answers are not rivals. My reading is that the earlier one is the purpose and these three are the working knowledge that serves it: a practitioner cannot see where the business is badly served by a rule without knowing why the rule does what it does, and cannot do anything about it without the means to change it.

Less stewarding, more reading what the rules find

The answer went on to what happens to the steward's job:

Further, I would suggest that Practitioners need to focus on other aspects of their work as the ‘Stewardship’ responsibilities will be reduced through automation. Analysis, trend identification and reporting, however.

The word is reduced, not removed, and the sentence that follows is a list I left without a verb. I have argued that the steward's skills move from spotting routine match patterns to research and analysis, from rote work to bespoke work, and that the number of people doing stewarding as a whole job could conceivably fall (Heads Up). This is the same expectation seen from the side of policy rather than matching.

My reading is that an automated rule produces something a manual programme rarely had: a complete record of every refusal and every exception. That record is the material for the three things in the list. Analysis is working out why a rule fires: a producing system that has changed, a definition that no longer fits the business, a process that never followed the policy. Trend identification is seeing which rules fire more each month and which exceptions keep being granted, which is where a policy and the business it governs have drifted apart. Reporting is putting that in front of whoever owns the decision, so the owner decides whether to change the rule, the process or the source. That is stewardship's judgement without stewardship's queue, and it is the work that keeps a policy adopted rather than merely enforced.

The same query

The last paragraph of my answer went to reporting, and to the claim that natural-language querying will make technical specialists unnecessary:

Despite all of the ‘natural language query’ “intelligence” being built into the ‘next generation of systems,’ it will still require technical specialists to backstop and double-check the AI as well as standardize the reports so the entire Enterprise is operating off of the same queries instead of relying on their own individualized versions.

This is the second objection from earlier, and the evidence on it is moving fast, so it is worth stating carefully. When Juan Sequeda, Dean Allemang and Bryon Jacob tested GPT-4 on enterprise questions over an insurance database in 2023, it "achieves an accuracy of 16%," and "this accuracy increases to 54% when questions are posed over a Knowledge Graph representation" [12]. When Fangyu Lei and colleagues built Spider 2.0, a benchmark of "632 real-world text-to-SQL workflow problems," the best system they tested "successfully solves only 21.3% of the tasks," against 91.2% on the original Spider [10]. The public leaderboard for that benchmark, as of 1 November 2026, shows engineered agent systems scoring 96.70 on its Snowflake setting and 76.23 on its broader "Lite" setting [11]. The settings differ, so the comparison is indicative rather than like-for-like, but it is a large improvement, and anyone arguing that machines cannot write enterprise queries is arguing against the trend.

It does not answer the question in the quotation, for two reasons. The systems at the top of that leaderboard are built and tuned by specialist teams; they are an argument for technical specialists, not against them. And a benchmark has one correct answer per question, fixed in advance. An enterprise does not. Its problem is that sales, finance and the warehouse each have a correct answer to "how many active customers do we have", and the answers differ. The semantic-layer tools exist to fix exactly that. dbt Labs describes its semantic layer's primary value as "to centralize and bring consistency to your metrics across your organization," allowing teams "to define metrics once, centrally" [13]. My reading is that defining a metric once is the reporting side of the same fight. Somebody has to decide which definition becomes the one the enterprise's queries run against, and somebody has to own changing it. A model that checks or corrects records rather than writing queries needs the same thing: it can apply a definition very well, but where the definition is contested, somebody still decides which one it applies. I have argued that a department's report may keep its own number when it can explain why and how the number differs (Two Models and a Seam), and that a semantic layer makes answers exact and repeatable without making them true (Precise and Wrong). Natural-language query makes asking cheap. It makes the question of whose query is the standard more urgent, because everyone can now produce their own version in seconds.

Where the argument goes next

Over the next decade the systems will do far more of the refusing, and that is mostly good. Rules that nobody disputes will stop being argued about, and records that break them will stop getting in. What the systems will not do is decide who is responsible for fixing the source or what the record should become. Those decisions will be made by whoever owns the rule, approves the model's suggestion, grants the exception and sets the query everyone else runs. Ownership and responsibility are still the ground the argument is fought on. My reading is that a practitioner who wants a policy adopted rather than merely enforced should spend less time on the queue and more on the rule: know why it fires, be able to change it, and know which side of the line each approval sits on today.

References

[1] Gartner, Inc., "Gartner Predicts 80% of D&A Governance Initiatives Will Fail by 2027, Due to a Lack of a Real or Manufactured Crisis," press release, 28 February 2024 (Saul Judah and Anurag Raj quoted). Read in full 2026-11-01. An analyst prediction; the release states no method. https://www.gartner.com/en/newsroom/press-releases/2024-02-28-gartner-predicts-80-percent-of-data-and-analytics-governance-initiatives-will-fail-by-2027-due-to-a-lack-of-a-real-or-manufactured-crisis-

[2] Zhamak Dehghani, "Data Mesh Principles and Logical Architecture," martinfowler.com, 3 December 2020. Read in full (governance section) 2026-11-01. https://martinfowler.com/articles/data-mesh-principles.html

[3] Bitol (Linux Foundation AI & Data), "Data quality," Open Data Contract Standard v3.2.0. Read in full 2026-11-01. https://bitol-io.github.io/open-data-contract-standard/latest/data-quality

[4] Kyverno project (CNCF), "Policy Exceptions," Kyverno documentation. Read in full 2026-11-01. Cited as an example of how a policy engine structures exceptions; the domain is Kubernetes, not data. https://kyverno.io/docs/exceptions/

[5] Databricks, "Attribute-based access control (ABAC)," Unity Catalog documentation, last updated 22 October 2026. Read in full 2026-11-01. https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac

[6] Emily M. Powers, Richard N. Shiffman, Edward R. Melnick, et al., "Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review," Journal of the American Medical Informatics Association 25(11):1556–1566, 2018. Read: the AHRQ PSNet summary only (the full text was not reachable). https://psnet.ahrq.gov/issue/efficacy-and-unintended-consequences-hard-stop-alerts-electronic-health-record-systems

[7] Lisanne Bainbridge, "Ironies of Automation," Automatica 19(6):775–779, 1983. Read in full (hosted copy) 2026-10-18; re-checked 2026-11-01. https://ckrybus.com/static/papers/Bainbridge_1983_Automatica.pdf

[8] Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 14, "Human oversight." Read in full (Article 14 only) 2026-10-18. Applies to high-risk systems. https://artificialintelligenceact.eu/article/14/

[9] Malcolm Hawker, "The Data Hero Summit 2026 – That's a Wrap!," Profisee blog, 9 October 2026. Read in full 2026-10-18. The author works for Profisee; our practice is a Profisee partner and earns a referral fee on a licence sale it refers, so this source's interest and ours run the same way. https://profisee.com/blog/the-data-hero-summit-2026-thats-a-wrap/

[10] Fangyu Lei, Jixuan Chen, Yuxiao Ye, et al., "Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows," ICLR 2025. Read: abstract and front matter, 2026-11-01. https://proceedings.iclr.cc/paper_files/paper/2025/file/46c10f6c8ea5aa6f267bcdabcb123f97-Paper-Conference.pdf

[11] Spider 2.0 leaderboard, spider2-sql.github.io. Read 2026-11-01; the page dates no entries. https://spider2-sql.github.io/

[12] Juan Sequeda, Dean Allemang and Bryon Jacob, "A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases," arXiv:2311.07509v1, 13 November 2023. Read: abstract, 2026-11-01. https://arxiv.org/abs/2311.07509v1

[13] dbt Labs, "dbt Semantic Layer FAQs," dbt documentation. Read in full 2026-11-01. https://docs.getdbt.com/docs/use-dbt-semantic-layer/sl-faqs