Judge’s ‘AI hallucination’ warning raises questions over government AI governance

A senior immigration judge has warned that the Home Office may have relied on an AI “hallucination” when refusing an asylum claim, in a case that is likely to reignite debate about governance and human oversight as artificial intelligence (AI) is deployed across government.

First reported in The Guardian, the case concerns a Moroccan woman whose asylum application was rejected using a reference to what officials described as a Country Policy Information Note supporting the decision. However, during an Upper Tribunal hearing, the judge found that no such document appeared to exist.

In his judgment, the judge said the refusal letter “bears hallmarks consistent with the use of artificial intelligence” and suggested the reference may have resulted from an AI hallucination – where a generative AI system produces inaccurate or fabricated information. He said that, if this proved to be the case, it would represent “an extremely serious failing” by the Home Office.

The judge compared reliance on the missing document to relying on “bogus evidence” and ruled that it amounted to a procedural irregularity. The asylum case itself remains ongoing, and neither the tribunal nor the Home Office has concluded that AI was definitively responsible for the error.

For public sector data and digital leaders, the significance extends beyond immigration policy. Government departments are increasingly adopting generative AI to summarise documents, search knowledge bases and support decision making. The Home Office itself has previously trialled AI tools to summarise asylum interviews and search country policy information, while maintaining that final decisions remain with human caseworkers.

With ministers positioning AI as central to improving productivity and service delivery, public bodies face growing pressure to demonstrate that governance and assurance are keeping pace with deployment.