Data Quality in the Era of AI Agents

TL;DR. Any data-intensive project faces, to some extent, risks from poor data quality. Now AI agents carry that risk across the whole company and beyond its boundaries, because they act on data, often with nobody watching. They are fed by internal structured data, documents and content, but also by external feeds, market data and the outputs of other agents, much of it owned by someone else. ‘Garbage in, garbage out’ used to cost a wrong report or a failed project; it now costs wrong actions, repeated at machine speed and passed from agent to agent. A company needs to know the quality of everything its agents consume, their context quality, and to check everything they produce. While AI models are black boxes, their inputs and outputs can be captured and systematically analysed through a quality and reliability lens. That is the basis of what I call agent assurance, a new form of quality monitoring for agentic AI.

Why Data Quality Matters More for Agentic AI

Over my career I have delivered more than 80 data-intensive projects across industries: data warehouses, data integration, business intelligence (BI), content performance and predictive models. In most of them, data quality was the single most important success factor.

The data engineering was always demanding: heterogeneous sources, complex transformations, and issues the data owners did not know about. So every project started with a data quality assessment, which often changed its scope. But the work had clear edges: it could be scoped, packaged and tested. The sources were known and inside the company, the checks were deterministic and kept reliable by automated tests, and the people reading the reports knew the business well enough to spot what the checks missed.

In earlier articles, including Data Quality in the Era of Artificial Intelligence (2017), I stressed the importance of data quality and why it matters most in the era of automated decisions. Those automated decisions are now smarter, real-time and in many cases autonomous, made by AI assistants, copilots, retrieval-augmented generation (RAG) systems and agents that take actions in other systems. And the edges have gone. An agent reads whatever it is given or allowed to find: tables, documents, emails, drafts, market data, web pages and the outputs of other agents. It combines them, decides, acts, and reports back to a person or hands its result to the next agent.

Bad data on a dashboard misleads people, who still have time to question it. Bad data in an agent turns into action, often before anyone notices.

Agents break the old way of working in two more ways. Agents are probabilistic: the same input can lead to a different answer or action, so a test that passes once is evidence, not proof. And their inputs are dynamic: a new tool, connector, search result or uploaded document can change what an agent reads at any point, without a release or a test. Data quality stops being a project with a start and an end, and becomes something the system checks every time an agent reads or acts.

The research points the same way. Gartner predicts that through 2026 organisations will abandon 60% of AI projects that are not supported by AI-ready data, and its survey of 1,203 data management leaders found that 63% do not have, or are not sure they have, the right data management practices for AI. A 2025 Harvard Business Review Analytic Services survey of 603 technology decision-makers found that only 6% trust AI agents to run core end-to-end business processes autonomously.

The research measures the problem. The rest of this article sets out the framework I have been developing to manage it: context quality across three kinds of input, a reputation ledger for every input, input reputation, agent health, and agent assurance, which brings them together.

From Data Quality to Context Quality

Data quality has many aspects: accuracy, completeness, consistency, validity, timeliness and uniqueness. Put simply: data quality is how reliable and usable data is for the purpose it serves. The more formal definition I have used for years is the level of compliance of a data set with contextual normality: data is good when it fits what is normal for the business, in the context where it is used.

That normality is set by rules, some defined by people and some derived statistically from the data itself. The rules depend on context: they reflect particular business processes, corporate knowledge and conditions, so the same property of the same entity can have different validation rules in different companies, markets, languages or regions.

AI adds one more piece of context: the decision the data feeds, often made in real time, and what is at stake when it is wrong. The same record, document or piece of content can be good enough for a monthly report and risky as the source an agent uses to answer a customer. A price list that is a week old is a footnote in an analysis and a wrong promise in a chat. So for AI systems I would extend the definition with five properties:

• Freshness: is this still true, and is it the current version?
• Provenance: where did it come from, and can we show it?
• Permission: who is allowed to see it, and through which assistant or agent?
• Verification: does it state facts, such as events, figures or claims, and have they been checked to the level the decision requires?
• Feedback: has a person or an agent flagged it as wrong, out of date or incomplete, directly or through a failure it contributed to?

Classic data quality is about structured and semi-structured data. Agents need it to cover everything they consume: an idea researchers call information quality, and which, for agents, I call context quality, the measured, evidence-backed quality of everything that enters an agent's context window. It follows from my definition of data quality above: quality was always judged in context, and for an agent the context is, literally, what it reads.

Agents consume three kinds of input, and context quality covers all of them:

• Structured data: records in databases, tables and feeds.
• Content: documents, reports, pages, messages, notes and drafts.
• Tool results: what agents fetch through tools at the moment of use: web search results, market data, third-party feeds, public facts and the outputs of other agents.

The rules differ for each kind of input: a record is checked against its schema and its history, a document against its status and its date, and a fact from a web search against several sources with a known track record.

Garbage In, Garbage Out: From Wrong Reports to Wrong Actions

‘Garbage in, garbage out’: flawed input produces flawed output, unless something checks and corrects it on the way in. The idea is almost as old as computing. For most of that time the garbage came out as an artefact, usually a report, and people had time to review it, fix it or reject it. That has changed, in five ways.

The consumer changed. Reports were read by people who knew the business and could question an odd number. Now the first reader is an agent that reads whatever it is pointed at, curated or not, and the person who receives its answer often lacks the context to question it.

The loop got shorter. Agents decide and act in seconds, many times a day, often without review. One data issue can become hundreds of wrong actions before anyone looks.

Failures became fluent. Bad data in a report shows up as an odd spike that makes an analyst ask questions. In an AI system, data quality issues can hide behind confident, well-written answers, so poor data no longer looks poor.

Agents hand over to agents. In complex workflows one agent gathers, another decides, a third acts. At each hand-off the source, the date and the doubt fall away, and new errors can creep in, so an uncertain input arrives at the end as a confident fact.

The boundary moved. Agents use data the company does not own: supplier feeds, market prices, public registers, news and web pages. No internal data quality programme has authority over any of it.

Consider a subscription business. A customer who signed up three weeks ago asks the support assistant whether they can cancel and get a refund. The search index holds last year's terms of service, with a refund within 30 days, next to the current terms, with no refund after 14 days, and nothing to say which is current. The assistant answers from the old version. A billing agent processes the cancellation and confirms the refund in writing. A week later, finance reverses it under the current terms. The customer has heard two different answers from one company, and every agent did its job correctly. The example is simplified for illustration: real cases involve longer chains of agents, many more sources, and errors that are far harder to trace.

The cost has moved too, from inside the company to outside it. Bad data in a warehouse cost trust among colleagues. Bad data behind a customer-facing assistant or a billing agent costs the trust of customers and partners, and can create commitments the business never meant to make.

How AI Agents Fail: Data, Content and Tool Results

AI agents can fail in many ways: flawed reasoning, a wrong tool call, a misread instruction, a security breach. Problematic input is one of the most important classes, and often the hardest to see, because the agent does everything right with the wrong material. Each kind of input fails in its own ways, and each needs its own quality indicators, recorded with the input and carried with it to the point of use.

Structured data fails in the classic ways: missing or invalid values, unexpected or corrupted formats, duplicates, inconsistent references and units, wrongly mapped properties and incomplete loads. Agents tolerate some of these better than the old pipelines did, because a model can read a date or an amount written in almost any format. Others become more dangerous: a model tends to fill a missing value with a plausible guess, and duplicates can crowd other sources out of what it retrieves. The indicators are validity, completeness, consistency and freshness, measured against validation rules and a profiled baseline.

Content adds failures of its own: stale documents that still rank high in retrieval, several versions of a policy with no sign of which is current, drafts and informal messages read as decisions, access rules lost on the way into the index, and missing dates and owners, so the system cannot tell what applies where. Worst of all is relevant content that is never retrieved, because nobody notices that the answer is incomplete. The indicators are status (draft, approved or superseded), authority, age, owner, access rules, and whether its facts have been checked.

Tool results come from sources the company does not control: feeds that change or stop without notice, market data and public facts that are wrong or out of date, other agents' outputs passed on without their sources or their uncertainty, and sources that disagree with no rule for which one wins. The indicators are each source's track record, update rhythm, coverage, rights of use and agreement with other sources; for another agent's output, also the inputs behind it and whether anything checked it.

The three combine. A small gap in a table, a stale document and a feed that has quietly stopped can each be harmless alone, and together turn a right answer into a wrong one. The root causes are familiar: software bugs, broken processes, and format changes at the source or the target. Two others are far more common now: a source with no owner, and a source outside the company's control. Nobody keeps the first current, nobody inside the company can keep the second current, and nobody notices when either stops being true.

In an AI system the symptoms look like this: answers that quote last year's policy, price or organisation chart; two agents that give different answers to the same question; evaluation scores that drop after a data change when the model did not change; confident answers to questions the data cannot answer; and users who quietly stop trusting the assistant and go back to asking a colleague. That last one is the failure I saw in BI projects years ago: users who do not trust the data gradually abandon the system, and the programme misses every target it was built for.

A Data Quality Strategy for AI Agents

With agents, that assumption covers every input, including the ones the company does not own. The six steps below are what AI-ready data (for agents, agent-ready data) means in practice.

1. Know your sources

For every source an agent may read, document what it contains, where it is stored, the time frame it covers, how and how often it is updated, which systems or people feed it, its known issues and limitations, its data model, and the stakeholders who understand it best. Add two fields that matter more for AI: who may see it, and who owns it. For a source outside your control, also record what you know about its reliability and what the agents should do when it fails.

2. Profile before you build

Data profiling describes the data with basic descriptive statistics and summaries, and records the result as a baseline. Identify the key entities and events, choose the typical time frame, look at trends, seasonality and peaks, summarise the distribution of each important property, and review a few outliers with the data owner. For documents, profile the collection too: how many versions, how old, how many without an owner or a date. For external feeds, profile their behaviour over time: delays, gaps and revisions. Automate the profiling where you can, and keep the report as the reference point for every later check.

3. Keep the rules outside the code

I have long recommended a data quality reference store: one place that holds the metadata and validity rules about your data, with fixing directives, available to every process that loads or reads data. Rules can be written by people, derived statistically, or both, and they should be dynamic rather than hard-coded. The same store now serves the agents: it tells them which sources and records are fit to use, for which tasks, and under which conditions.

4. Ground the agents in inputs that pass the checks

Grounding means the system answers from sources you can name, not from whatever the model happens to remember. Make it strict. Use only sources that pass your validation rules. Carry the source, date, owner and quality indicators with every input, so each agent, and the agent after it, can weigh a fresh, owned, checked source above a stale or unverified one. Show the sources with every answer. Retire content that has passed its freshness rule. And design the system to say that it does not know when grounding is missing, rather than produce something plausible.

5. Evaluate every time the data changes

Most teams re-test an AI system when they change the model or the prompt. Fewer re-test it when the data changes, which happens far more often. Keep a set of questions with known, correct answers and the sources that should support them, written with the people who own the data, and run it on every significant data release. A right answer from the wrong sources, or a missing source, is a warning too. Track the results next to an overall index of data quality, weighted by what matters: a wrong transaction record or a wrong policy is far more important than a broken link to an image.

6. Give every source an owner

Data ownership is the step most AI programmes skip, because it is organisational rather than technical. Every source an agent can read needs a named owner who is accountable for its accuracy and freshness, who receives the alerts when checks fail, and who signs off the evaluation questions for their domain. For an external source, the owner is the person accountable for the decision to use it. Without an owner, every other step decays.

Modern systems must be aware of the quality of the data they consume, and able to identify, report and handle poor data before it reaches a decision.

The Black Box Has Inputs and Outputs: How to Monitor AI Agents

A language model is a black box: nobody can fully explain how it reached a particular answer. But every agent has inputs and outputs, and both can be recorded, measured and acted on. This is the basis of what I call agent assurance: an always-on system that watches the quality of what every agent reads and produces, and acts before a problem reaches a customer.

Both the agents and the company should know, at all times, what each agent used, what it referenced, and what it should have used but missed. I recommend a central monitoring function for all of a company's agents, built on four records:

• Inputs: what each agent read, from which source, with which quality indicators at the time.
• Outputs: what it answered, decided or did, and whom it handed the result to.
• The links between them: which inputs lay behind each output, across the whole chain of agents.
• What was missed: relevant sources that existed but were never retrieved or used, found through evaluation questions, broader searches and people's corrections.

With these records the company can answer the questions that matter when something goes wrong, and before it does. Which decisions used a source that has since been found faulty? Which agents read superseded content this week? Which useful sources do agents never reach? Where do agents disagree with each other, or with the people who review their work? When a source fails a check, the monitor finds every agent and every decision it touched.

The monitor checks outputs too: against the rules in the reference store, against known answers, and against each other. An output that breaks a rule is held before it reaches a customer or another agent.

Then it closes the loop. It knows about quality issues, trends and sudden changes, and how critical each one is, so it decides who to notify, through which channel, and whether to keep serving the data with a warning or stop serving it until it is fixed.

Input reputation

The capability I see as essential for the agentic era is a reputation ledger: a companion asset for every input an agent can use, whether a table, a document, a feed or another agent's output. It holds two things:

• Access history: which agents used it, when, for which tasks, and what they produced from it.
• Quality signals: direct ones, when an agent or a person reports it as stale, contradictory or incomplete, and indirect ones, when it was one of the inputs to an operation that failed or was flagged.

Indirect signals are the new part. A failure is logged anyway, because a customer complains, a person overrules the agent, a check rejects the output or the agent reports a problem; the ledger marks every input behind it. One failure proves nothing: the input may have been fine and the agent wrong. A pattern does. The superseded terms of service behind most of the reversed refunds would stand out within days, and go to their owner with the evidence attached. Outputs that prove right add credit in the same way. From its ledger, every input earns what I call its input reputation, and the agents' everyday work becomes a continuous measure of the quality of what they read.

Agent health

The same records make prediction possible. Every agent depends on a known set of sources, and every source carries its quality indicators, so the monitor can score each agent's health from the quality of what it reads: fresh or stale, complete or patchy, consistent or contradicted, owned or orphaned, trusted or often implicated in failures. When a source an agent depends on degrades, the agent's health falls, and its predicted performance falls with it, before a single wrong answer reaches a customer. Checking outputs tells you what went wrong; the quality of the inputs tells you what is about to.

Put together, these scores make a dashboard of agent health: every agent, the sources it depends on, their quality today, its predicted performance, the tasks it stopped, whether it is paused, and what it reported. It gives the people who run an AI programme the view of their agents that a data team has of its warehouse.

AI may be a black box, but what goes into it and what comes out of it are not.

AI Agents as Data Quality Sensors

There is an opportunity here too: AI agents can improve data quality as well as depend on it. Agents read more of a company's data, content and tool results than any person does, and they read it all day. Designed with the right rules, they can measure the quality of what they read as they read it: an expired date, two documents that contradict each other, a feed that disagrees with the contract, a figure outside its normal range. Each observation goes into that input's reputation ledger, and to its owner.

A support assistant that finds two versions of the terms of service, one approved and one superseded, does not pick at random. It answers from the approved version, flags the conflict and records it, and the owner of the terms receives one alert instead of a stream of complaints.

An agent built this way knows the quality of its own inputs, and it knows when to stop. When an input it needs fails a check and it cannot tell which version is right, it aborts the task, says why, and reports the source to its owner, rather than producing something plausible. In the cancellation case, where neither version was marked current, the support assistant would have stopped before promising a refund, told the customer that a colleague would confirm the terms, and sent both versions to their owner.

The rules do not come with the agents; they have to be designed in, from the same reference store that serves the rest of the pipeline. A company can go further and run dedicated monitoring agents, independent of the agents that do the work. They track quality metrics in structured data and check the facts that matter in the market the company operates in: prices, regulations, competitors' offers and the claims made about its products.

Your Company Is Data for Other People's Agents

The boundary runs the other way as well. Customers, partners, investors and job candidates now ask AI assistants about companies and their products, and the assistants answer from what they find: the company's website, public registers, reviews, news, comparison sites and old press releases. Buying agents compare suppliers on the prices and terms they can read. If that information is outdated, inconsistent or wrong, other people's agents will repeat it, fluently, to the people whose decisions matter to you.

So data quality now has an external side: the facts about your company that other agents consume. Treat them as a source you own. Keep them consistent across your own channels, correct what is wrong elsewhere where you can, and check what the main assistants say about you, the same way you check your own agents' outputs.

Opening your systems to agents through MCP

The exposure grows when a company opens its own systems to agents. MCP, the Model Context Protocol, has become a common way to connect agents to tools and data, and through MCP servers and agent-facing APIs a company can let its customers' and partners' agents query its catalogue, prices, stock, orders and documentation directly. From that moment, internal data quality is part of the product. A wrong stock level or an outdated price becomes a tool result inside another company's agent, which acts on it at machine speed, with your company as the source.

Those agents will judge your server the way this article suggests you judge your own inputs. A server that returns stale or inconsistent data builds a poor input reputation in other people's agents, and gets used less, or passed over for a competitor's. So before exposing a system, apply the same discipline as for your own agents: give every tool an owner, return each result with its source, date and quality indicators, monitor what other agents ask for and what they receive, and treat the server as a product with a quality promise, not a technical connector.

Who Should Own Data Quality in an AI Programme?

Treat data quality as part of every AI use case, not as a separate clean-up project that never gets funded. When you assess a use case, assess the inputs behind it: their owners, their quality baselines, their freshness, and which of them the company does not control. When you budget a use case, budget the work to make its inputs fit. The central monitoring of agents needs an owner too, at the level that owns the AI programme, because it is where the risks of every use case meet.

The tools are arriving in three separate markets, sold to three different buyers: data quality and observability platforms for the data team, agent monitoring and evaluation platforms for the engineers, and AI visibility tools for marketing. Each covers part of the same problem. The problem needs one owner.

In my experience, this is where AI strategies quietly succeed or fail, long before the choice of model matters, and it is one of the first things I look at in an AI strategy engagement.

For the cultural side, where people learn to trust and use data, see Data-Driven Companies and the Importance of a Data Culture. For where data-driven companies go next, see Harry Mamangakis on how to transform a company into an AI-powered organisation.

The models will keep getting better on their own. The data will not.

An earlier version of this article, Data Quality in the Era of Artificial Intelligence, was published on Medium in September 2017.

References and further reading

DAMA UK (2013). The Six Primary Dimensions for Data Quality Assessment. DAMA UK Working Group.

Gartner (2025). Lack of AI-Ready Data Puts AI Projects at Risk. Press release, 26 February 2025.

Harvard Business Review Analytic Services (2025). From the Edge to the Core: Bringing Agentic AI to the Heart of the Enterprise. Research report, sponsored by Workato.

Krasadakis, G. (2017). Data Quality in the Era of Artificial Intelligence. Medium.

Wang, R. Y. and Strong, D. M. (1996). Beyond Accuracy: What Data Quality Means to Data Consumers. Journal of Management Information Systems, 12(4), 5–33.