Data modernisation for government: from legacy data to data products

The problem: decisions built on stale data

Government decisions on budgets, services and policy depend on data that often arrives weeks after the period it describes. This is not a minor operational inconvenience. For the UK Government, the gap between when economic activity happens and when reliable statistics become available means decisions as significant as the Budget are often taken using information that is already out of date.

The Office for National Statistics (ONS) publishes key economic indicators such as GDP, employment and public debt with delays ranging from two to seven weeks, while early estimates are often revised as additional data becomes available. The table below illustrates the approximate publication lags for several important indicators.

These delays matter in normal circumstances. During periods of heightened volatility or geopolitical shocks, they can significantly reduce the effectiveness of traditional forecasting methods.

The stopgap: nowcasting and its limits

To bridge the gap between when data is collected and when it is published, organisations including the Bank of England use nowcasting: producing early estimates of economic activity by combining high-frequency indicators such as survey data, card payments and other real-time measures before official statistics become available.

Nowcasting improves the timeliness of decision-making, but it has important limitations.

• Transitory volatility. Unexpected events such as strikes, climate incidents or geopolitical shocks can create temporary spikes in data that models interpret as longer-term trends rather than one-off events.
• Structural breaks. Major events such as the COVID-19 pandemic fundamentally changed patterns of economic activity, making historical models less reliable until they adapted to new behaviours.
• Representativity bias. High-frequency data and surveys may not accurately reflect the wider population. The challenges experienced by the ONS Labour Force Survey illustrate how declining response rates can reduce confidence in official statistics and delay the transition to more representative data collection methods.

These examples are not outliers. They reflect a broader challenge facing government organisations that depend on multiple data sources which are often neither as timely nor as reliable as the decisions they are intended to support.

The underlying issue: a legacy data architecture problem

The root cause is not poor data collection but legacy data architecture. Government organisations frequently hold data that is siloed within departmental systems, inconsistently documented and lacking sufficient metadata, making it difficult to reuse and share.

The Central Digital and Data Office (CDDO) has acknowledged that government lacks a consistent framework for data ownership, with unclear roles, accountabilities and responsibilities limiting effective cross-government data sharing.

This aligns with the government’s own assessment in Transforming for a Digital Future: 2022 to 2025 Roadmap for Digital and Data, which identifies inconsistent data quality, limited data sharing and outdated legacy systems as key barriers to digital transformation.

Modernising government data therefore requires more than replacing legacy technology. It requires a different approach to managing data itself, one that supports legacy to cloud migration, modern governance and trusted data sharing.

What is a data mesh, and why does it need governance?

One approach that has gained significant traction in both the public and private sectors is the data mesh. Defined by Zhamak Dehghani in 2019, data mesh is a sociotechnical approach to decentralised data architecture that shifts responsibility for analytical data from a central team to the domain teams that know it best, supported by a shared, self-service platform.

For government, the appeal is clear. Rather than relying on central IT to own and maintain every dataset, departments become accountable for publishing trusted, well-governed data products that can be reused across government. The CDDO’s data ownership model reflects this approach through defined roles for Data Owners, Data Stewards and Data Custodians.

However, organisations do not need to adopt a fully decentralised data mesh to benefit from these principles. The core concepts of data products and data contracts can be implemented within more centralised architectures.

Without shared governance, any architecture – whether decentralised or centralised—risks recreating the silos it is intended to remove. The answer is a data product approach underpinned by robust data contracts.

Data products and data contracts: the foundation of trust

A data product is a reusable, self-contained package that combines data, metadata, semantics and templates to support multiple business use cases. A data contract is a formal agreement between data producers and consumers that defines the quality, structure, semantics and availability of that data. Unlike a legal contract written in plain language, a data contract is written in code, typically YAML or JSON, making it enforceable through automation rather than manual processes.

Data contracts provide the governance that enables organisations to manage data as a trusted product. They define schema, data quality rules, service level agreements (SLAs), access controls and infrastructure information, ensuring data is discoverable, consistent and reliable.

For government organisations, this directly addresses some of the most fundamental questions: when will data be available, in what format, and who is accountable if it does not meet the agreed standard?

By creating an auditable, version-controlled specification that both producers and consumers can rely on, data contracts help establish trust across departments while reducing duplication and inconsistency.

This aligns closely with the CDDO’s guidance on making essential shared data assets (ESDAs) available across government. The guidance requires datasets to meet the government data quality framework, align with minimum metadata standards and be published through APIs that meet government standards. Functionally, this is the same discipline that data contracts enforce: documented, quality-assured, access-controlled data with clear ownership.

Towards a data-driven organisation: Aker Systems’ approach

This is the context in which Aker Systems works as a trusted partner to the UK Government. Our expertise lies in designing and implementing regulated data platforms and secure cloud architecture for highly sensitive environments, including Critical National Infrastructure and national security, where data integrity, availability and operational resilience are paramount.

The objective is not to replace existing data sources but to help organisations transition from batch-based, informally governed data towards a data product-oriented architecture that delivers measurable benefits: increased trust in data, reduced time to publish new datasets, the removal of data redundancies and silos, and the foundation for a secure, AI-ready data platform. This transformation often forms part of a wider cloud migration strategy for modern government organisations.

The UK Government’s guidance on making datasets ready for AI states that the effectiveness, safety and legitimacy of AI adoption are fundamentally constrained by the quality, structure and governance of underlying data. AI-ready data must be accurate, complete, consistent, secure and enriched with metadata so it can be trusted by both people and machines.

Organisations that attempt AI without this foundation face a well-documented challenge. As the National Audit Office has highlighted, providing raw data or basic APIs without information on data quality or provenance can lead to misunderstanding and misuse, particularly where AI systems operate without sufficient context.

Aker’s delivery model

Aker’s delivery model is built around four practical stages that help organisations transition to a modern, data product-oriented architecture.

Define a self-service platform. The central IT team stops being a data bottleneck and instead provides the shared infrastructure that enables domain teams to publish, transform and govern their own data without requiring deep infrastructure expertise. This supports a modern cloud operating model while addressing one of the key challenges identified in the CDDO roadmap: siloed development leading to inconsistent levels of digital maturity across government.

Write and enforce data contracts. Every data product is governed by a contract defining schema, semantics, service levels, lineage, access controls and change-management rules. Version control ensures schema changes are introduced in a way that gives downstream consumers time to adapt, improving consistency and reducing the risk of conflicting outputs across departments.

Deploy the data platform. Whether cloud-based, built on a hybrid cloud architecture, or on-premises, the platform provides the shared capabilities needed to publish, discover and access trusted data products. A data catalogue and access portal enable consumers to find, understand and use the data they need without specialist knowledge of the underlying infrastructure. This supports the government’s own ambition for a cross-government Data Marketplace.

Aker’s Cloud Enablement Platform (CEP) and Data Enablement Platform (DEP) provide the technology layer that underpins these capabilities, supporting an operational resilience architecture for mission-critical public services while enabling regulated cloud migration and enterprise cloud migration programmes.

Operate and improve continuously. A data product is never finished. As source data, consumer requirements and methodologies evolve, data products must be continuously monitored, improved and governed. This includes maintaining service levels, gathering user feedback and ensuring interoperability as new domains are added. Aker builds these monitoring and feedback loops into delivery from the outset rather than treating them as a retrofit.

Connecting data modernisation to AI readiness

An important objective of modern data ownership is the shift from treating data as an asset to treating it as a product, requiring organisations to understand how data is used, where it creates value and how it is governed throughout its lifecycle.

This is also the foundation for AI. The government’s AI-readiness framework states that datasets must be technically optimised, meet defined quality standards, comply with legal and regulatory requirements, and be managed responsibly throughout their lifecycle.

A data product-oriented architecture, governed through data contracts and delivered via a self-service platform, directly supports these principles. It also provides the foundation for cloud-native transformation while aligning with the UK Government’s AI Opportunities Action Plan, which recognises that AI-driven public value depends on accessible, consistent, high-quality and trustworthy data.

For government organisations, data modernisation is therefore not simply about replacing legacy technology. It is about creating trusted, governed data products that enable better decision-making today while providing the foundation for responsible AI adoption tomorrow.