Risk

Prohibited risk

• Art. 5: certain AI based practices shall be prohibited in the EU in their entirety. These are systems used for:
  • social scoring
  • cognitive behavioural manipulation
  • real-time remote biometric identification (unless if used for targeted searches for specific potential victims of crime)
  • creating or expanding facial recognition databases through untargeted scraping of facial images from the internet or CCTV footage
  • emotion recognition

High risk systems

Art. 6

• Mandatory requirements:

  • providers of high-risk AI systems must comply with mandatory requirements and undergo ex-ante conformity assessment
  • obligations include establishing quality and risk management systems, creating detailed technical documentation, and ensuring compliance with data requirements

• Technical requirements:

  • High-risk AI systems must generate operation logs for traceability and incorporate features for user oversight
  • Transparency in operation is crucial for user interpretation and appropriate use
  • Non-EU providers must appoint an authorized representative in the EU if directly providing AI systems to the EU market

• Art. 6 and Annex III: focus on regulating high-risk AI systems that pose potential threats to health, safety, or fundamental rights.

• AI systems that pose a significant risk to health, safety or fundamental rights, including in the following categories:

  • medical devices
  • vehicles
  • emotion-recognition systems
  • law enforcement

• Categories of High-Risk AI Systems:

  • integrated into safety-critical products like medical devices, lifts and certain vehicles
  • standalone systems with fundamental rights implications (such as those used in critical infrastructure management and educational assessment

• Criteria for High-Risk Determination

  • Annex III lists AI systems considered high risk, with exceptions for systems performing narrow or preparatory tasks
  • Providers must document assessments and notify high-risk systems to the EU database

Annex III - High Risk systems

• Non-banned biometrics
  • remote biometric identification systems, excluding biometric verification that confirm a person is who they claim to be
  • Biometric categorisation systems inferring sensitive or protected attributes or characteristics
• Critical infrastructure
  • Safety components in the management and operation of critical digital infrastructure, road traffic and the supply of water, gas, heating and electricity
• Education and vocational training
  • AI systems determining access, admission or assignment to educational and vocational training institutions at all levels
• Employment, workers management and access to self-employment
  • AI systems used for recruitment or selection, particularly targeted job ads, analysing and filtering applications, and evaluating candidates
• Access to and enjoyment of essential public and private services
  • AI systems used by public authorities for assessing eligibility to benefits and services, including their allocation, reduction, revocation or recovery
  • Evaluating creditworthiness, except when detecting fraud. Evaluating and classifying emergency calls. Risk assessments and pricing in health and life insurance
• Law enforcement
  • Evaluating evidence reliability during criminal investigations or prosecutions
  • Assessing an individual’s risk of offending or re-offending not solely based on profiling or assessing personality traits or past criminal behaviour
• Migration, asylum and border control management: Polygraphs
  • Assessments or irregular migration or health risks. Examination of applications for asylums, visa and residence permits
  • Detecting, recognising or identifying individuals, except verifying travel documents