AI has rapidly moved from experimentation to expectation. But while organisations are racing to deploy models and agents, many are discovering a harder truth. AI doesn’t create agility – it exposes whether it exists in your data layer.
According to research from Nexer Group, more than 80% of AI projects fail. From our perspective, it’s not the model that breaks, but the underlying data estate. Fragmented schemas, inconsistent structures and legacy artefacts make it difficult to produce output that is reliable, explainable, or even relevant.
When organisations merge or transform systems, they often uncover years of accumulated inconsistencies: custom tables, redundant fields, conflicting data definitions and so on. These issues can directly impact latency, reporting accuracy and the ability to generate real-time insights.
Left unresolved, these problems compound and at scale, they undermine accuracy and create operational risk right across the business. Decision-making slows, costs increase and the organisation loses the ability to respond quickly, which is exactly what AI is supposed to enable.
In practice, AI performance is bound by the quality and structure of the data it’s trained on and how well internal teams can collaborate. If data is incomplete, duplicated, or misaligned across systems, your models will inherit those issues and exacerbate them.
So, the question is: how do teams start to tackle underlying data challenges?
What business agility means in the age of AI
Business agility is often framed as an organisational capability. For example, how quickly can the business react and stabilise in the face of changing market conditions, legal requirements (M&As or cutoffs), or regional governance and industry regulations?
From a technical standpoint, business agility comes down to how quickly you can query, trust and act on your data. This means being able to find the right datasets among thousands if not hundreds of thousands of tables. It also requires being able to understand the lineage and context of the data and that’s before you add on the final layer of deploying AI models (LLMs or SLMs), ideally without compromising business critical processes.
If your data lake is too tightly coupled, poorly documented, or fragmented across incompatible systems, your organisation is constrained from the start. Those that suffer from these qualities need to prioritise the establishment of a clear, structured, flexible data architecture.
Preparing your data for AI
In 2025, Gartner analysts predicted that this year organisations will abandon 60% of AI projects due to a lack of AI-ready data.
As systems evolve, requirements shift, or new business objectives are realised, data often starts to accumulate without being organised. This is often called unstructured enterprise data, e.g. emails and logs that are harder to standardise and it only adds to the complexity of data migration projects to prepare for AI.
The first step to understanding your data landscape is to look at the whole picture – not just the primary systems, but everything connected to them. This includes legacy tables, unused schemas and data flows that may no longer reflect how your business operates.
From there, the goal is to design data structures that align with the current operational reality. This means standardising schemas, removing duplications and creating a clear framework. This allows you to ensure and monitor consistent data definitions across all your business systems.
Critically, this process isn’t just about cleaning data, it’s about making it reliable and usable. LLMs and SLMs both require data that is accurate and structured in a way that can be easily interpreted across use cases, e.g. internal agent chat queries.
Deciding what data to keep or archive
A persistent misconception about AI-readiness is that more data improves outcomes. In practice, often the opposite is true. According to research from Proofpoint, 41% of enterprises with more than 10,000 employees are managing more than a petabyte of data – 1,000 terabytes. Much of it is redundant or obsolete, including the ‘dark data’ that many organisations fail to even recognise, let alone use.
This is putting pressure on cloud services, not to mention the associated costs. Aside from the clear security risks involved, obsolete or redundant data will only introduce more noise into AI models. Training AI models on these datasets increases the likelihood of inaccurate output, model drift and higher computational costs.
This makes data curation a very critical next step once you have your structure in place. Businesses should strive to identify datasets that are relevant to current operations, consistent in structure and sufficiently recent to reflect real-world behaviour and use cases. A well-curated dataset will deliver better results than a large unfiltered one. The best thing to do is create a separate dataset to test it so other business-critical services aren’t disrupted.
Once teams know which dataset gives the best response or drives better organisational efficiencies, it is easier to decide which segments are needed most and which can be relegated to the archives.
Knowing where to standardise or customise
Standardisation is essential for scalability. It reduces complexity, improves operations and makes data easier to process across interconnected systems.
However, excessive standardisation can remove differentials that may benefit your business in areas like customer engagement, pricing strategies, or market-specific operations. So, what can businesses do to overcome this challenge?
Ideally, they should standardise at the system level while still preserving differentiation at the business logic level. In practice, this means standardising core processes that don’t change often, such as finance, supply chain and order management, while retaining flexibility in areas where that competitive advantage is most likely to be felt, like marketing or sales.
Taking this approach allows organisations to maintain clean, scalable data architectures without losing the nuance that drives business and brand value.
Building an AI-ready foundation
Becoming AI-ready isn’t a one-off transformation. AI highlights data problems and organisations that treat data as a byeproduct will struggle to scale AI effectively.
Those that treat data as a managed and structured business asset will be able to adapt, iterate and compete. This will require continuous governance, regulation, oversight, accountability and ongoing management of the data estate, which is where many fall short – focusing on deploying models rather than maintaining the systems those models depend on.
Business agility in the age of AI is about data integrity and how quickly you can access and act on it. If organisations lack clear ownership, consistent governance and continuous monitoring, all data issues will persist and AI will amplify them. Those that rise to these challenges will stand above their competitors.
By Peter Bosley-Sharpe, President EMEA at SNP Group.
- Data & AI