The Public Law Project (PLP) has criticised the Department of Work and Pensions (DWP) for failing to deliver on its promise of transparency around it use of AI and automation.
DWP wants to increase reliance on AI and automation, including for detecting fraud and error in social security claims and payments.
PLP said DWP’s annual report and accounts for 2023-24, published on 22 July 2024, speaks generally about its use of data analytics and machine learning, including safeguards in place. However, PLP said the report provides “no meaningful information about what technologies DWP is piloting or using nor how they are, or will be, deployed.”
PLP also said that it provides “little information about DWP’s own assessment of the potentially discriminatory impact that the technologies might have on protected groups or vulnerable claimants.”
PLP said this “weak compliance” with the Public Accounts Committee (PAC) recommendation to report to Parliament on an annual basis on the impact of data analytics on protected grounds and vulnerable claimants is “very disappointing”. It said, “there is also scant detail on the safeguards that DWP is purportedly adopting to mitigate harm.”
Additionally, “the DWP has provided no real detail to allow meaningful scrutiny of the impact of the tools it is adopting. There is a real risk that (as identified by the National Audit Office) technologies of this nature may lead to discriminatory outcomes by unfairly targeting particular claimants, including because of their protected characteristics.”
Although the DWP’s report does refer to a fairness analysis it has carried out in relation to the Universal Credit (UC) Advances model, the results have not been published. The report states that the results ‘do not present any immediate concerns of discrimination, unfair treatment or detrimental impact on customers’.
The DWP has not provided any details of its assessment due to concerns that publication would allegedly ‘allow fraudsters to understand how the model operates’.
PLP said transparency around automation and AI matters because such technologies are currently being used to process UC Advances and to investigate or suspend UC payments, and without transparency, “we cannot hold the DWP to account if it is using those technologies unlawfully.”
PLP is currently acting for the Work Rights Centre (WoRC) in relation to a potential claim against the DWP regarding its use of automated technologies in triaging claims for UC Advances for enhanced review prior to payment, and identifying UC payments for investigation and suspension.
“PLP and WoRC consider that DWP’s lack of transparency and current deployment of such technologies may be unlawful for a number of reasons, including that their use does not appear to accord with published guidance and may be contrary to data protection law, human rights law and equalities law,” it said in a statement.
PLP sent an initial pre-action letter to DWP on behalf of WoRC earlier this year, raising eight potential grounds of challenge.
“We continue to engage in pre-action correspondence with DWP, with the aim of learning more about how precisely it is using these technologies and what safeguards it is adopting to mitigate harm,” it said.
