What is a high-risk AI system?
By Joey van Vliet, founderUpdated
In short
A high-risk AI system, under the EU AI Act, is an application that can significantly affect people’s health, safety or fundamental rights, such as AI in recruitment, credit scoring or access to essential services. These systems are subject to strict requirements on risk management, data quality, logging and human oversight.
Also known as: high-risk AI
Two kinds of high-risk system
Annex III, standalone systems: AI in areas the law names explicitly. Recruitment and other decisions in the employment relationship, access to education, credit scoring, access to essential public and private services, law enforcement, migration and certain forms of biometrics.
Annex I, built into products: AI as part of products already covered by EU product safety rules, such as medical devices, machinery, lifts and toys.
The obligations apply in phases, and the dates differ per kind. Check the current dates in the consolidated text of the AI Act on EUR-Lex before you plan.
What it means if you only use the system
Even as a deployer, so when you don’t build the system yourself, high-risk applications come with extra duties. You use the system according to the instructions, assign human oversight to someone with the knowledge and authority to intervene, keep the input relevant and representative, retain the logs and let the people affected know that a high-risk system is being used.
That’s clearly more than for ordinary AI applications, but still far less than what the provider has to do.
Why it matters in practice
Go through Annex III with one question: does this AI make or support a decision about a person that affects their work, money or access to something?
For many SMEs, the answer is yes exactly once, and it’s about recruitment software. If you have AI pre-sort CVs or score candidates, you’re in this category, even if the supplier doesn’t make a point of it. More in CV screening with AI.
The sensible order: ask your supplier in writing whether the system is classified as high-risk and how they’re preparing for the obligations. That answer determines whether you need to set something up yourself or can build on what the provider supplies.
Frequently asked questions
Which high-risk applications are most common in SMEs?
Mainly AI in recruitment: filtering CVs, scoring or ranking candidates. Beyond that, AI that helps decide on credit or on access to essential services. Most other Annex III categories, such as biometrics and critical infrastructure, don’t affect the average SME.
When do the high-risk obligations apply?
In phases, and the dates differ for Annex III systems and for systems built into Annex I products. Use the consolidated text of the AI Act on EUR-Lex as your source, not a summary from a year ago.
