By Naresh Nayar, Rick Hamilton and Jaswant Singh
Why AI models stall between proof of concept and production, and what technology leaders can do about it.
The Problem Isn’t the Model
A customer-facing AI agent confidently answers a benefits question. The answer is wrong because it retrieved a superseded policy document from an unversioned, access-uncontrolled corpus. The business now has three concurrent problems: customer harm, regulatory exposure, and an internal investigation that cannot reproduce the retrieval context that produced the response. No one can say which version of which document the model saw, because provenance was never captured. The investigation drags on for weeks; the root cause, ungoverned data, remains in place for the next incident. In AI-enabled enterprises, this is not an edge case. It is the predictable outcome of deploying AI on a data substrate built for reporting, not autonomous action. Models are only as trustworthy as the data they ingest; in most organizations, that data is neither traceable enough to explain nor governed well enough to defend.
In 2024, one of the authors was serving as CTO of a healthcare research organization when the team deployed its first RAG solution. We anticipated the usual technical challenges: problematic chunking strategies, improper ranking algorithms, nonfunctional requirements like system performance. The biggest problem, however, was none of these. It was out-of-date and contradictory data sources, resulting in the system misrepresenting current scientific thinking and organizational policy. The technical architecture worked, but the data substrate beneath it was ungoverned. Now amplify this lesson across an enterprise deploying autonomous agents that depend on data sources spanning dozens of systems and domains, whose immediate responses and downstream decisions may never be reviewed by a human. The problems we expected in 2024 were largely architectural. The problem that actually mattered was upstream, as ungoverned, conflicting data quietly degraded output quality as the AI became more relied upon and essential to our business.
In this context, data governance and modern data management together form the trust architecture for AI-driven analytics and automation. Governance defines decision rights, standards, and accountability for data meaning, quality, provenance, and access. These policies and standards do not enforce themselves; they must be translated into technical controls that operate within the data ecosystem. Modern data management enforces those standards as controls across pipelines, catalogs, and APIs, and ideally, it produces an evidence trail that makes outputs explainable and defensible. When governance and management are disconnected, AI scales faster than trust, and small data defects become systemic failures.
Governance structures and operational risk frameworks define who is accountable and what must be monitored. This paper addresses an important question previously raised: whether the data infrastructure beneath those frameworks is engineered to make accountability and monitoring possible? For related concepts, see AI Governance is Broken: Here’s How to Fix It and AI Risks Don’t Wait for Committees.
The central problem for most enterprises is not a lack of data, but rather is the lack of executable governance: policies that are enforced automatically at the point of data movement and model use, rather than documented in frameworks that nobody operationalizes. Until governance is implemented as enforceable controls within systems, AI will continue to scale faster than trust. In working with organizations across healthcare, financial services, and insurance, we have found three structural failures that recur with striking consistency, and they persist not because leaders are unaware of data governance, but because their governance programs are designed to produce documentation rather than controls.
Together, we explore this important topic more thoroughly in the full Substack article, including why this matters now, the executable governance model, how to start and metrics that matters. Read the full article here.