Modern data sourcing: From tech sprawl to trusted, purpose-driven data

The public sector has never had access to more data – from administrative systems, commercial feeds and sensors to long-standing survey series. But turning that abundance into better public services is far from automatic.

That topic sat at the heart of a panel on modern data sourcing at the recent Think Data for Government event in Westminster, featuring contributions from Giuseppe Sollazzo, deputy director and head of data enablement at DWP Digital, and Steve Whiting, CEO of Agile (pictured).

For Sollazzo, modern data sourcing begins before anyone touches a database.

“Modern data sourcing is about the ability of asking questions before even we get to the data engineering aspect,” he said.

One DWP Digital example is finding people who might be eligible for a certain benefit, so the organisation can write to them, he said. That, he stressed, is very different from a technocratic hunt for more data.

Instead, teams need “repeatable processes”, even where some steps remain manual. Generating national statistics, for instance, “often requires people to follow some recipes. They know what the data is, but we need to go get a new one and rebuild the pipeline.”

Across DWP Digital, he sees “the full spectrum: one-offs, repeatable, automated” sourcing patterns – and that spectrum effectively maps to a maturity journey.

“As we go towards automated, we get the more mature approach,” he said. “We know what the stuff is. We know what the process has done.”

From spreadsheets to embedded data platforms

That maturity shows up in the tools and practices around data sourcing.

Often, Sollazzo said, it begins with “information asset owners, Excel spreadsheets”, then progresses into “the famous core drilling versions of data capitals… data capital, which we put everything in – almost as a “geological tool to understand the history of data.”

But the real step change comes when those tools are “embedded into business processes”.

“That’s where we simplify things and data sourcing – good data sourcing – becomes almost second nature, key to the business, not to the data,” he said. The goal is to move away from standalone data projects towards sourcing that is simply how the organisation works.

One example is in housing benefit. One recent transformation, he said, “was able to bring the data source into a single housing benefit system… allowing people to be paid the right benefit, but also to allow local authorities to recoup any overpayment.” The impact was stark: “from six weeks to 30 minutes.”

Tackling tech sprawl and building simple, shared architectures

For his part, Whiting looked at the foundations across government. “When we stand up and look at the whole, I can see a number of foundational things… maturity, culture, trust, capability,” he said – all of which must be in place for modern sourcing to succeed.

One growing problem, especially as organisations adopt cloud services and AI tools, is “technology sprawl”.

“What we used to see is data sprawl, and more recently we’re seeing a technology sprawl,” Whiting warned. “Lots and lots of types of technology… for ultimately very similar goals.”

He pointed to hyperscalers’ converging technology stacks as a way through, with “data ecosystems, data fabrics… to store data and ultimately build in AI-native capabilities.” The public sector, he suggested, needs “standardised, simplified architecture” to “break out some of the barriers” that currently make reuse and sharing harder than it should be.

Democratising data – and doing federation for real

Looking back over the last few years, Whiting sees a pattern: “There was a move towards optimisation with data. Then there was a move to sharing data.” That has led to “products like data marketplaces, exchanges, and the ability to serve up data which is trusted, has value and is a reusable sense of information.”

Asked about the much-discussed idea of data federation, he argued that “curating trusted sources of information and making them available through government marketplaces or global marketplaces is, by its nature, creating federation.” The challenge now is less technical and more about adoption and use.

Both panellists agreed that good sourcing is inherently multidisciplinary. “Data is a technical topic… but those people shouldn’t be the people understanding the problem,” said Sollazzo. In large organisations, “we have people who are experienced at certain types of policies… people who have worked on the frontline… people who understand the legal aspects of data sharing.”

For Whiting, one personal non-negotiable is mindset: “We need digital expertise capability, but we also need curiosity… If you haven’t got any curiosity, you lose the passion.”

AI readiness: don’t create tomorrow’s technical debt

No panel on data in 2025 can avoid AI, and this one was no exception.

Whiting framed the issue as “AI readiness”: “Are organisations ready to deploy AI safely? Is their data trusted? Have they got the right processes?” He linked this back to GDPR as a foundation, but argued that organisations also need to understand “how well governed data is, how you understand its lineage, that whatever solutions you’re bringing in aren’t creating technical debt for the future.”

“With AI, we’ve really hit the accelerator, and things are moving so fast,” he said. That means leaders must “architect and design solutions that are ahead of the curve for tomorrow, otherwise you’re just creating technical debt for the future.”

Sollazzo stressed that many basic principles still apply. AI, he argued, “cannot be realised without measuring and doing things as we would do for any other entity.” He also highlighted a classic public sector issue: “Business data that is collected for one purpose doesn’t necessarily fit another.” That makes responsible reuse for AI a design problem, not a quick win.

Above all, modern data sourcing must remember who it is for. “We need to make sure that the benefit is accessible by the public as well as the organisation,” said Sollazzo.

Added Whiting: “The end is that we deliver valuable services for UK taxpayers and citizens.”