Article 60: Testing of High-Risk AI Systems in Real World Conditions Outside AI Regulatory Sandboxes
Article 60
Testing of High-Risk AI Systems in Real World Conditions Outside AI Regulatory Sandboxes
Updated on 31 July 2024 based on the version published in the Official Journal of the EU dated 12 July 2024 and entered into force on 1 August 2024.
1. Testing of high-risk AI systems in real world conditions outside AI regulatory sandboxes may be conducted by providers or prospective providers of high-risk AI systems listed in Annex III, in accordance with this Article and the real-world testing plan referred to in this Article, without prejudice to the prohibitions under Article 5.
The Commission shall, by means of implementing acts, specify the detailed elements of the real-world testing plan. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 98(2).
This paragraph shall be without prejudice to Union or national law on the testing in real world conditions of high-risk AI systems related to products covered by Union harmonisation legislation listed in Annex I.
2. Providers or prospective providers may conduct testing of high-risk AI systems referred to in Annex III in real world conditions at any time before the placing on the market or the putting into service of the AI system on their own or in partnership with one or more deployers or prospective deployers.
3. The testing of high-risk AI systems in real world conditions under this Article shall be without prejudice to any ethical review that is required by Union or national law.
4. Providers or prospective providers may conduct the testing in real world conditions only where all of the following conditions are met:
- the provider or prospective provider has drawn up a real-world testing plan and submitted it to the market surveillance authority in the Member State where the testing in real world conditions is to be conducted;
- the market surveillance authority in the Member State where the testing in real world conditions is to be conducted has approved the testing in real world conditions and the real-world testing plan; where the market surveillance authority has not provided an answer within 30 days, the testing in real world conditions and the real-world testing plan shall be understood to have been approved; where national law does not provide for a tacit approval, the testing in real world conditions shall remain subject to an authorisation;
- the provider or prospective provider, with the exception of providers or prospective providers of high-risk AI systems referred to in points 1, 6 and 7 of Annex III in the areas of law enforcement, migration, asylum and border control management, and high-risk AI systems referred to in point 2 of Annex III has registered the testing in real world conditions in accordance with Article 71(4) with a Union-wide unique single identification number and with the information specified in Annex IX; the provider or prospective provider of high-risk AI systems referred to in points 1, 6 and 7 of Annex III in the areas of law enforcement, migration, asylum and border control management, has registered the testing in real-world conditions in the secure non-public section of the EU database according to Article 49(4), point (d), with a Union-wide unique single identification number and with the information specified therein; the provider or prospective provider of high-risk AI systems referred to in point 2 of Annex III has registered the testing in real-world conditions in accordance with Article 49(5);
- the provider or prospective provider conducting the testing in real world conditions is established in the Union or has appointed a legal representative who is established in the Union;
- data collected and processed for the purpose of the testing in real world conditions shall be transferred to third countries only provided that appropriate and applicable safeguards under Union law are implemented;
- the testing in real world conditions does not last longer than necessary to achieve its objectives and in any case not longer than six months, which may be extended for an additional period of six months, subject to prior notification by the provider or prospective provider to the market surveillance authority, accompanied by an explanation of the need for such an extension;
- the subjects of the testing in real world conditions who are persons belonging to vulnerable groups due to their age or disability, are appropriately protected;
- where a provider or prospective provider organises the testing in real world conditions in cooperation with one or more deployers or prospective deployers, the latter have been informed of all aspects of the testing that are relevant to their decision to participate, and given the relevant instructions for use of the AI system referred to in Article 13; the provider or prospective provider and the deployer or prospective deployer shall conclude an agreement specifying their roles and responsibilities with a view to ensuring compliance with the provisions for testing in real world conditions under this Regulation and under other applicable Union and national law;
- the subjects of the testing in real world conditions have given informed consent in accordance with Article 61, or in the case of law enforcement, where the seeking of informed consent would prevent the AI system from being tested, the testing itself and the outcome of the testing in the real world conditions shall not have any negative effect on the subjects, and their personal data shall be deleted after the test is performed;
- the testing in real world conditions is effectively overseen by the provider or prospective provider, as well as by deployers or prospective deployers through persons who are suitably qualified in the relevant field and have the necessary capacity, training and authority to perform their tasks;
- the predictions, recommendations or decisions of the AI system can be effectively reversed and disregarded.
5. Any subjects of the testing in real world conditions, or their legally designated representative, as appropriate, may, without any resulting detriment and without having to provide any justification, withdraw from the testing at any time by revoking their informed consent and may request the immediate and permanent deletion of their personal data. The withdrawal of the informed consent shall not affect the activities already carried out.
6. In accordance with Article 75, Member States shall confer on their market surveillance authorities the powers of requiring providers and prospective providers to provide information, of carrying out unannounced remote or on-site inspections, and of performing checks on the conduct of the testing in real world conditions and the related high-risk AI systems. Market surveillance authorities shall use those powers to ensure the safe development of testing in real world conditions.
7. Any serious incident identified in the course of the testing in real world conditions shall be reported to the national market surveillance authority in accordance with Article 73. The provider or prospective provider shall adopt immediate mitigation measures or, failing that, shall suspend the testing in real world conditions until such mitigation takes place, or otherwise terminate it. The provider or prospective provider shall establish a procedure for the prompt recall of the AI system upon such termination of the testing in real world conditions.
8. Providers or prospective providers shall notify the national market surveillance authority in the Member State where the testing in real world conditions is to be conducted of the suspension or termination of the testing in real world conditions and of the final outcomes.
9. The provider or prospective provider shall be liable under applicable Union and national liability law for any damage caused in the course of their testing in real world conditions.
Table of Contents
Chapter III: High-Risk AI Systems
- Section 1: Classification of AI Systems as High-Risk
- Section 2: Requirements for High-Risk AI Systems
- Article 8: Compliance with the Requirements
- Article 9: Risk Management System
- Article 10: Data and Data Governance
- Article 11: Technical Documentation
- Article 12: Record-Keeping
- Article 13: Transparency and Provision of Information to Deployers
- Article 14: Human Oversight
- Article 15: Accuracy, Robustness and Cybersecurity
- Section 3: Obligations of Providers and Deployers of High-Risk AI Systems and Other Parties
- Article 16: Obligations of Providers of High-Risk AI Systems
- Article 17: Quality Management System
- Article 18: Documentation Keeping
- Article 19: Automatically Generated Logs
- Article 20: Corrective Actions and Duty of Information
- Article 21: Cooperation with Competent Authorities
- Article 22: Authorised Representatives of Providers of High-Risk AI Systems
- Article 23: Obligations of Importers
- Article 23a
- Article 24: Obligations of Distributors
- Article 25: Responsibilities along the AI Value Chain
- Article 26: Obligations of Deployers of High-Risk AI Systems
- Article 27: Fundamental Rights Impact Assessment for High-Risk AI Systems
- Section 4: Notifying Authorities and Notified Bodies
- Article 28: Notifying Authorities
- Article 29: Application of a Conformity Assessment Body for Notification
- Article 30: Notification Procedure
- Article 31: Requirements Relating to Notified Bodies
- Article 32: Presumption of Conformity with Requirements Relating to Notified Bodies
- Article 33: Subsidiaries of Notified Bodies and Subcontracting
- Article 34: Operational Obligations of Notified Bodies
- Article 35: Identification Numbers and Lists of Notified Bodies
- Article 36: Changes to Notifications
- Article 37: Challenge to the Competence of Notified Bodies
- Article 38: Coordination of Notified Bodies
- Article 39: Conformity Assessment Bodies of Third Countries
- Section 5: Standards, Conformity Assessment, Certification, Registration
- Article 40: Harmonised Standards and Standardisation Deliverables
- Article 41: Common Specifications
- Article 42: Presumption of Conformity with Certain Requirements
- Article 43: Conformity Assessment
- Article 44: Certificates
- Article 45: Information Obligations of Notified Bodies
- Article 46: Derogation from Conformity Assessment Procedure
- Article 47: EU Declaration of Conformity
- Article 48: CE Marking
- Article 49: Registration
Chapter V: General-Purpose AI Models
Chapter VI: Measures in Support of Innovation
- Article 57: AI Regulatory Sandboxes
- Article 58: Detailed Arrangements for, and Functioning of, AI Regulatory Sandboxes
- Article 59: Further Processing of Personal Data for Developing Certain AI Systems in the Public Interest in the AI Regulatory Sandbox
- Article 60: Testing of High-Risk AI Systems in Real World Conditions Outside AI Regulatory Sandboxes
- Article 61: Informed Consent to Participate in Testing in Real World Conditions Outside AI Regulatory Sandboxes
- Article 62: Measures for Providers and Deployers, in Particular SMEs, including Start-Ups
- Article 63: Derogations for Specific Operators
Chapter VII: Governance
Chapter IX: Post-Market Monitoring, Information Sharing and Market Surveillance
- Section 1: Post-Market Monitoring
- Section 2: Sharing of Information on Serious Incidents
- Section 3: Enforcement
- Article 74: Market Surveillance and Control of AI Systems in the Union Market
- Article 75: Mutual Assistance, Market Surveillance and Control of General-Purpose AI Systems
- Article 76: Supervision of Testing in Real World Conditions by Market Surveillance Authorities
- Article 77: Powers of Authorities Protecting Fundamental Rights
- Article 78: Confidentiality
- Article 80: Procedure for Dealing with AI Systems Classified by the Provider as Non-High-Risk in Application of Annex III
- Article 81: Union Safeguard Procedure
- Article 82: Compliant AI Systems which Present a Risk
- Article 82a
- Article 83: Formal Non-Compliance
- Article 84: Union AI Testing Support Structures
- Article 79: Procedure at National Level for Dealing with AI Systems Presenting a Risk
- Section 4: Remedies
- Section 5: Supervision, Investigation, Enforcement and Monitoring in Respect of Providers of General-Purpose AI Models
- Article 88: Enforcement of the Obligations of Providers of General-Purpose AI Models
- Article 89: Monitoring Actions
- Article 90: Alerts of Systemic Risks by the Scientific Panel
- Article 91: Power to Request Documentation and Information
- Article 92: Power to Conduct Evaluations
- Article 93: Power to Request Measures
- Article 94: Procedural Rights of Economic Operators of the General-Purpose AI Model
Chapter XIII: Final Provisions
- Article 102: Amendment to Regulation (EC) No 300/2008
- Article 103: Amendment to Regulation (EU) No 167/2013
- Article 104: Amendment to Regulation (EU) No 168/2013
- Article 105: Amendment to Directive 2014/90/EU
- Article 106: Amendment to Directive (EU) 2016/797
- Article 107: Amendment to Regulation (EU) 2018/858
- Article 108: Amendment to Regulation (EU) 2018/1139
- Article 109: Amendment to Regulation (EU) 2019/2144
- Article 110: Amendment to Directive (EU) 2020/1828
- Article 111: AI Systems Already Placed on the Market or Put into Service and General-Purpose AI Models Already Placed on the Marked
- Article 112: Evaluation and Review
- Article 113: Entry into Force and Application