Data readiness: what your data needs before any ML project

Every machine learning proposal contains a sentence like “we have a lot of data”. It is almost always true and almost never the relevant question. What matters is whether the data can answer the specific question the project exists to answer, and that is a much narrower test.

What the evidence actually supports

There is a well-travelled statistic that 85% of machine learning projects fail. It is repeated constantly and sourced thinly, and it belongs in the same category as the claim that late defects cost a hundred times more to fix: directionally believable, numerically folklore. We would rather give you the figures that have a traceable origin.

That second number is the one worth sitting with. It is not a survey of laggards. It is a survey of the people whose job is data management, and most of them could not say with confidence that their practices were adequate for what the business was about to ask of them.

The reframe that saves the most money

Gartner’s own definition of AI-ready data is worth reading closely: data aligned to specific use cases, governed at the asset level, supported by automated pipelines with quality gates, managed through live metadata, and continuously quality-assured.

The first clause is doing the work. Readiness is not a property a data estate has. It is a relationship between a dataset and a question.

The five questions

1. Does it exist, in one place? Not “do we collect it”. Is it retrievable by a system rather than a person. Data spread across six spreadsheets, a legacy database and one colleague’s judgement exists in the sense that matters to the business and not in the sense that matters to a model.

2. Is it labelled, and by someone who knew the domain? For anything supervised, label quality is a hard ceiling on model quality, and no amount of modelling gets above it. The common failure is not missing labels, it is labels applied inconsistently by different people over several years against a definition that quietly drifted.

3. Is it governed? Someone owns each significant field and can say what it means, when its meaning last changed and what a null in it signifies. The absence of this is what produces models that learn an artefact of a 2021 process change and then fail quietly in production.

4. Is it connected? A pipeline, not a quarterly export. A model that trains on a snapshot someone emailed is a demonstration. Getting from there to production is mostly this problem, and it is engineering rather than data science.

5. Is it measured? Automated checks on completeness, ranges and distributions that fail loudly and block the pipeline. Without these, the way you find out the upstream schema changed is that predictions get strange three weeks later.

“We don’t have enough data” is usually about the wrong task

This objection ends more projects than it should, because the volume required collapsed over the last few years for a large class of problems. It depends entirely on what you are asking for.

If the goal is to let people ask questions of your own documents and contracts and policies, you need no labelled examples. You need documents that are current, findable and permissioned correctly. That is a different readiness problem, and a more tractable one. The architecture choice behind it is covered in RAG, fine-tuning or prompting.

If the goal is genuinely predictive, forecasting demand, scoring risk, predicting churn, then the volume question returns and so does every one of the five above. Being clear which kind of project you are proposing is the cheapest thing you can do at the start.

Three blockers that are not about volume

• Labels that disagree with themselves. Three people classified support tickets over four years against a definition that changed twice. The labels exist and they are worse than none, because they teach the model the inconsistency. Re-labelling a sample and measuring agreement between annotators is the cheapest diagnostic available and almost nobody runs it.
• Permissions discarded at ingestion. The data is fine and the access rules were left behind in the source systems. This does not surface as a data problem, it surfaces as a security incident. We cover the mechanism in building an AI feature on your own data without leaking it.
• No path from the model to a decision. The prediction is accurate and nothing in the business changes as a result, because no process consumes it. This is the most common reason a technically successful pilot produces nothing, and it is not a data problem at all.

What to do before committing budget

1. Name one use case, precisely. Not “use AI in operations”. One decision, made by a named person, that a system could inform.
2. Work backwards to the data it needs. Which fields, from which systems, at what freshness, for which population. This usually takes a day and frequently ends the project, which is a good outcome at this price.
3. Pull a real sample and look at it. Not a schema. Actual rows, counting nulls in the fields that matter, checking date ranges, and finding out how many records fail a rule someone assumed always held.
4. If it is supervised, measure annotator agreement. Have two people independently label 100 records. If they disagree on a quarter of them, that is your ceiling and no model will exceed it.
5. Cost the pipeline, not the model. Getting the data flowing reliably is most of the work and most of the budget, and it is the part that keeps costing after launch.

This is a discovery phase with a specific shape, and the arguments for not skipping it are the same as everywhere else: see what actually happens in a discovery phase.

The question worth asking out loud

Does this need machine learning at all? A meaningful share of projects that arrive framed as prediction problems are answered better by a rule, a report or a better-designed form. Those are cheaper, explainable, testable with ordinary tools, and they do not need any of the five conditions above.

Being willing to reach that conclusion is what separates a data readiness assessment from a sales process. More on where the line sits in where AI actually pays off in business software, and on what it costs once it is running in what it costs to run an LLM feature in production.

Our AI and machine learning team runs this assessment as a fixed piece of work, and we will tell you when the answer is a report rather than a model. If you have a use case in mind and no idea whether your data supports it, describe the decision you want to improve.