ComputerWeekly

AI tool used in asylum case reviews failing accuracy tests


An artificial intelligence (AI) tool introduced to speed up the processing of asylum claims has been found to make errors during Home Office testing.

The Home Office introduced the Asylum Case Summarisation (ACS) tool in April 2026 to speed up decisions and reduce backlogs in asylum claims. Based on GPT-4o, the tool creates automated summaries of interviews with asylum seekers, which can be up to 50 pages long, before they are reviewed by decision makers.

According to Home Office ministers, the ACS saves caseworkers 23 minutes for each case they assess. But research from the Open Rights Group and MPs, first reported by the Independent, has revealed significant numbers of errors in the AI-generated summaries. Asylum applicants are not told that AI has been used in the consideration of their claims, ministers have confirmed.

Sample testing by Home Office experts has revealed that a significant number of AI-summarised interviews failed quality checks, according to documents obtained by MPs and the campaign group. Home Office subject matter experts have carried out dip samples of 203 summaries to test their accuracy, since the system was rolled out.

Summaries were marked as failed if they contained “multiple factual errors or misrepresentations”, or “significant inaccuracies, contradictions or misstatements”, according to documents obtained by Open Rights Group (ORG).

In the first stage of the review, 139 summaries passed, 19 were flagged up for further review and 45 (22%) failed. In the second stage review, 56 (15%) of the sample summaries failed.

However, AI-generated summaries that omit material facts about the case can still pass if the rest of the content is accurate. This is because the relevance of content included in the summaries is not listed as one of the grounds for assessing failure.

Only a small proportion of summaries checked by experts

Home Office experts have checked only a small proportion of the AI summaries generated by the tool between April, when it was rolled out, and July 2026.

According to the ACS data protection impact assessment (DPIA), an average of 17,515 asylum cases are processed each month. Out of an estimated 70,000 cases processed between April and July, only 203 were checked by subject matter experts, representing 0.3% of the AI-generated summaries.

The results of the dip-testing exercise have been “mainly discussed verbally” in a working group. Although an interim report on spot testing has been provided to an internal board, it has not yet been published.

Ministers have confirmed that AI summaries of cases are not considered during monthly quality-assurance checks. The Home Office has told MPs that it has no plans for an independent audit of the ACS.

Human in the loop?

The DPIA for ACS states that the tool was built to automate work previously conducted by technical specialists who reviewed and summarised cases before passing them on to a decision maker.

According to the ORG, this suggests that technical specialists no longer review the whole file before it is examined by a decision maker, effectively removing a layer of expert eyes from reviewing the case. The campaign group said it was hard to reconcile the time savings against claims by ministers that decision makers must also read every document.

The Asylum Case Summarisation tool was rolled out to all asylum caseworkers despite known errors. The Home Office’s evaluation of the pilot study found that 9% of summaries it produced were inaccurate, had missing information and had to be removed from the pilot.

Another AI tool, the Asylum Policy Search (APS), was rolled out to all asylum decision makers from July 2025. The tool, which has a chat-style interface for searching country policy information, aims to speed up the process of looking for relevant country policy information for a verdict.

More transparency needed over AI

Open Rights Group’s migrant digital justice programme manager Sara Alsherif said that the risks of using AI in asylum applications were high and the process should be more transparent.

“Deciding whether someone can or cannot seek refuge in the UK is one of the most serious and life-changing decisions that the state can make about an individual. The processes that lead to that decision must be transparent, fair and accountable,” she said. “But the Home Office continues to test AI on people seeking asylum in the UK without even telling them.”

A Home Office spokesperson said that all asylum verdicts were made by trained decision makers, not artificial intelligence: “Our AI tools support caseworkers by helping them navigate interview records and policy information, but they cannot be used in isolation and do not make decisions on claims.

“We regularly monitor the performance of these tools alongside our established quality-assurance processes. All asylum claims continue to be considered on their individual merits by trained decision-makers.”



Source link