Solving the gen AI data challenge

How do you get your data ready for gen AI? It’s a question being asked the public sector looks to take advantage of AI to improve productivity, reduce costs and enhance citizen services. Assuming appropriate use cases can be identified, will the quality of departmental data be up to scratch? What further work and safeguards are needed to ensure the benefits will outweigh the risks?

Speaking at the Think Data for Government event in London, Deepak Shukla, public sector GTM lead at Amazon Web Services (AWS) said a lack of data remains a challenge in the implementation of AI projects in the public sector, adding organisations still rely on spreadsheets and don’t have the data sets to train models and go into production.

“That is kind of challenging,” he said. “But on the other side… we are looking to use AI to improve data management practices. There are a lot of legacy applications. There’s a lot of data there, but we don’t know exactly what data exists within those applications.”

Shukla said AWS is testing using AI to interrogate databases and give recommendations, as well as helping build data engineering pipelines.

Solving problems at scale

Despite challenges around data, Jenny Brooker, chief data architect at the Central Digital Data Office said gen AI use cases are emerging.

Alongside the popularity of chatbots, she said there bigger problems being solved by AI, citing examples from the recent Civil Service Data Challenge. These include a geospatial planning tool for NHS district nurses to maximise home visits, a chatbot to improve policy summarisation for frontline staff, sharing of death data between the NHS and DWP to improve the bereavement and benefits process, and using data algorithms to optimise the management of prison space across the UK.

“There’s great use of POCs [proof of concepts],” she said, adding the CDDO wanted to ensure its AI framework promoted ethical and trusted use of data. “We’re going to get to the point where we’re just understanding how to start to really use this to solve those of problems at scale,” she said.

Dr Shruti Kohli, head of data science (innovations) at DWP said the department has been focusing on the control, integration and governance around its data. She emphasised the necessity of safeguards, data control, and the development of a policy framework.

“It started with enthusiasm and a mix of fear in the start, but overall in this journey, we have brought colleagues together … we are not just going to build a system and give it to them, we are going to build with them.”

The DWP is continuing to work with strategic governance colleagues on what data it needs, she said.

Shukla added that departments don’t necessarily need to change their data strategy, but the focus must be on quality. “If you’re going to feed your AI models with garbage, you’re going to [get] garbage out,” he said, adding organisations must be able to track the value of their data.

“It’s a good opportunity to use AI to bring the data agenda forward within the organization,” he said. “There’s a lot of enthusiasm, there are lot of business initiatives. Start with the business funding, look at what data you need… you need to look at the culture as well.”