Denmark stands against critics of public sector AI plans

Legal gap

Doubts were allayed and a legal gap verified in 2023 when Datatilsynet, the Danish Data Protection Agency (DPA), told Copenhagen municipality that existing law gave it a right to use personal data to train an AI system for triaging patients in need of physiotherapy interventions, but not the right to use it. The government drafted the bill to fill the gap. Digital rights campaigners protested that its general permission lacked measures to contain the risks.

“This is a law that says whenever AI is involved, normal rules don’t apply,” Jesper Lund, chairman of Danish legal campaign group IT-Politisk Forening, told Computer Weekly. IT-Pol’s consultation response said the proposed law was so offensive to data protection principles that it should not be tabled in parliament.

“The big irony is that the Danish Data Protection Agency has complained about public authorities using AI because the legal basis wasn’t precise and clear. The government’s response is to grant an even less precise and much broader legal basis. They have been given carte blanche to do what they want,” he said.

“In a rule-of-law society, public authorities cannot do whatever they want. It is not the same as for citizens and companies. For public authorities, anything that is not expressly permitted is not allowed, because we want people to be free of repression from the state,” Lund added.

“If the government has powers of discrimination over you, you should know what they are, and you should have options for redress. You do not want the government taking arbitrary action against you. That is why we have laws,” he said.

The big irony is that the Danish Data Protection Agency has complained about public authorities using AI because the legal basis wasn’t precise and clear. The government’s response is to grant an even less precise and much broader legal basis

IT-Pol and other legal campaign groups are concerned that the government has proposed a broad legal basis for public bodies to use AI in ways that interfere with people’s privacy rights, when European Union law requires such intrusions to have precisely defined circumstances, purposes and safeguards against abuse. It lacks sufficient limits and safeguards and neglects to guarantee that the general provision is necessary or proportionate, according to critics.

Datatilsynet produced a long list of warnings after the first draft bill was published in February. Crucially, the draft provided insufficient legal justification for derogations ministers and public bodies could make from European data protection (GDPR) rights protecting people from state intrusion. A revised draft the government published in September failed to answer most, a data protection official close to the situation told Computer Weekly. It had reflected some, the DPA said on 12 October, when the second consultation closed. An intervening general election annulled the first draft, forcing its reintroduction.

The DPA accepted the general legal basis but found inadequate justification for the bill’s proposed derogations, said the official. That was, notably, AI reuse of personal data collected for other purposes and delegated powers for ministers to exempt whole sectors or systems from obligations to meet people’s data-subject rights.

Rather than neglect the justification for derogating people’s rights, the bill left ministers and public bodies responsible for it. Ministers could justify derogations of data-subject rights only under GDPR Article 23 exceptions, such as public security, crime, health, social security or important financial matters. Failing that, public bodies would have to justify data reuse by doing compatibility assessments. They would still have to do assessments to meet their GDPR obligations anyway, as with any other risky data processing. They would still have to demonstrate a particular system’s processing was necessary and proportionate, and implement safeguards against abuse. The bill just gave them a basis to do that.

The DPA had effectively told the government it had solved a tiny part of the problem, but many remained, the official said. The regulator had meanwhile learned that many public bodies were ready to press go on AI projects once the bill passed, believing it did indeed give them carte blanche to run systems on personal data. So, it devised its consultation response half as a public service announcement to public bodies: the law did not make everything A-OK, and the buck stopped with them. Without an exception, ministers and public bodies would have to provide the missing justification.

Stalled projects

AI projects obstructed for want of legal cover included a system for child welfare cases, and health systems to improve cancer treatment, aid GP diagnosis and identify people at risk of hospitalisation, according to a 2024 analysis of 40 “signature projects” the government designated as test cases.

The government said last year that two-thirds faced legal difficulties, preventing four from being used at all, and others had limited functions. Of 31 health and social care projects, one was operational at the last assessment, and barely half were completed. Technical problems, including widespread poor data quality, often accounted for delays.

Denmark’s IT Industry Association, representing public sector AI suppliers, welcomed the bill, chief consultant Asbjørn Flügge told Computer Weekly. But industry was concerned that leaving responsibility for justifying AI use to ministers could cause fragmentation, with some parts of the public sector becoming more permissive than others.

Denmark, known as a world leader in digital public services, naturally hit these challenges before other countries, said Flügge, whose PhD concerned an AI project to predict long-term unemployment that stalled for want of legal basis. The government appears not to have thought things through, while aspects of GDPR have become outdated and unhelpful. Public opinion of AI has soured since February’s first draft.

“All government’s AI policies and hopes were blocked by its own lack of legal regulation,” said Ida Marie Schrøder, associate professor of technologies in practice at the IT University of Copenhagen, who co-wrote a study of a media storm that led to early Danish attempts to use AI in aid of child protection being cancelled. One, Gladsaxe, intended to detect vulnerable children, became a byword for unacceptable algorithmic practice.

Many of the signature projects were blocked because they were not legal to start with, said Schrøder. “That was lucky for us Danes because no one had a clue what they were developing. And I am afraid they still don’t,” she said.