Editorial

Unlocking efficiency through AI and open data in the UK civil service

Data specialists from the private and public sector explore how AI and open data can transform the UK civil service by driving efficiencies and enhancing decision-making.

Posted 10 December 2024 by Christine Horton


The UK government is increasingly looking to emerging technologies like artificial intelligence (AI) and open data to drive greater efficiencies across the civil service. But realising the full potential of these tools requires overcoming a range of technical, cultural and organisational challenges. That was a key takeaway from a panel discussion at Think Data for Government, which brought together senior data and AI experts from across the public and private sectors (pictured).

“There’s a lot of talk about AI pilots and proof of concepts, but we’re hearing very little about true operationalisation at scale,” said Sian Thomas, chief data officer at the Department for Business and Trade.

Thomas highlighted the difficulties in evaluating the impact of data and AI initiatives, particularly for open data where it can be hard to track how the information is being used.

“When you make data open, you don’t have a way of contacting users, so how can you quantify the value that people are getting from it?” she said.

The panel said government agencies need to carefully consider the rationale and quality criteria for their open data programs to ensure they are sustainable and trustworthy. “If you don’t engage the people who have been working in the trenches, who know where the bottlenecks are, then you risk creating tools that don’t actually help them do their jobs better,” warned Jeni Tennison, executive director at Connected by Data.

Tennison drew on her experience working on legislation.gov.uk, which used a combination of workflow management, basic natural language processing, and expert participation to tackle a massive backlog of legislative amendments.

“We really needed to respect the public sector workers and understand their expertise. Getting their early and continuous input was key to making sure the tools were fit for purpose,” she said.

Getting the data foundations in place

The panel also highlighted the importance of data quality and secure data sharing mechanisms in enabling effective AI adoption. Harneet Singh, senior specialist on data and AI for the UK public sector at AWS, emphasised the need to focus on getting the data foundations right before rushing to implement AI.

“Sometimes we struggle with just finding where the data is residing in our systems. So data discovery, people and process become overarching and important even before we think about technology,” Singh said.

Varun Khandelwal, principal AI specialist at Snowflake, pointed to examples like New Zealand’s national data platform, which brings together data from 29 different health entities to enable more holistic insights and decision-making.

“When you think about open data as ‘data products’ with clear user value, it changes the mindset. You start to consider things like who’s consuming it, how it’s being used, and how you can ensure continuity,” said Khandelwal.

The panellists agreed that addressing cultural barriers around data sharing and collaboration would be crucial for unlocking the full benefits of AI and open data in the public sector. “There’s a big cultural barrier to actually making these technical solutions work. We need to have a mindset across all the people involved in data to make sure there’s a willingness to share, to use the data, and to learn from those experiences,” said Thomas.

Tennison echoed this sentiment, noting that efficiency shouldn’t be the sole metric. “We need to think about relational services, diversity and inclusion – not just targeting throughput, but targeting quality of service.”

Event Logo

If you are interested in this article, why not register to attend our Think Data for Government conference, where digital leaders tackle the most pressing issues facing government today.


Register Now