I have reviewed hundreds of vendor contracts and due diligence files over the past decade. The pattern is almost always the same: a questionnaire goes out, the vendor fills it in, someone checks the boxes, the contract gets signed. Efficient. Documented. And almost entirely silent on the question (for more on what regulators actually expect, see our GDPR vendor management guide) that now matters most.
What AI systems is this vendor actually running? And what happens to my organisation when one of them fails?
Traditional vendor assessment was designed to answer a reasonable question: is this company stable, secure, and trustworthy enough to work with? That question is still valid. But it is no longer sufficient. More importantly, what have they not identified? Who is accountable when it causes harm? Does any of it meet the regulatory obligations that now attach to AI deployment across virtually every regulated sector? Most vendor questionnaires do not ask any of these. Most vendors, if I am being honest, have not prepared to answer them.
This article sets out the three-layer framework I use when advising clients on AI vendor risk: your own internal governance standard, the regulatory obligations that now apply under the EU AI Act, and the technical standards (ISO 42001 and NIST AI RMF) that translate those obligations into practice. I will also be direct about where this framework currently runs into its limits, because pretending otherwise does not help anyone who actually has to do this work.
The Evidence: What the AI Incident Data Tells Organisations About Third-Party Risk
TL;DRAI-related incidents hit a record 233 in 2024, up 56.4% year on year, with accuracy failures and safety events dominating the categories. The average data breach now costs USD 4.88 million, and only 24% of generative AI initiatives are secured. These figures apply directly to vendor-delivered AI: the risk does not stay inside the vendor's perimeter.
Start with the numbers, because they are striking. According to the Stanford HAI AI Index Report 2025, drawing on data from the AI Incident Database, 233 AI-related incidents were reported in 2024: a 56.4% increase over 2023, and a record high. By October 2025, that annual total had already been surpassed. Something is accelerating. The question is whether vendor assessment processes are accelerating with it.
The AI Incident Database categories illuminate where AI risk concentrates. Accuracy and reliability failures account for the largest single category, with 361 recorded incidents: cases where AI systems produced outputs that were incorrect, inconsistent, or fabricated. Safety incidents account for 268 records. Reports of malicious actors using AI have grown eightfold since 2022, according to the AI Incident Database. These are not abstract classifications. They represent real operational failures in systems that organisations were relying on, including systems delivered by third-party vendors.
The financial dimension is equally concrete. The global average cost of a data breach reached USD 4.88 million, a 10% increase over the prior year and the largest single-year jump since the pandemic. Only 24% of generative AI initiatives are secured, according to the same research. For vendor relationships where the vendor deploys AI in the delivery of services, the exposure is not contained within the vendor's perimeter. It extends to every organisation that relies on that service.
Half of Organisations Had a Negative AI Incident Last Year
TL;DRMcKinsey's 2025 survey found 51% of organisations experienced a negative AI incident in the prior year, including output inaccuracy, compliance violations, and privacy breaches. Fewer than half took concrete mitigation steps afterward. Regulators and litigation counsel increasingly treat that inaction as evidence of a governance maturity gap, not mere oversight.
That is the headline from McKinsey's 2025 State of AI survey of nearly 2,000 participants across 105 countries. Output inaccuracy, compliance violations, privacy breaches, reputational damage, AI systems taking unauthorised actions. Fifty-one percent. And of those organisations, fewer than half took concrete mitigation steps. Explainability (understanding why the system decided what it decided) remains the gap most organisations experience and fewest have controls for.
What strikes me about that McKinsey figure is not the 51%. It is the inaction that follows it. Organisations identified the risk. Then did nothing concrete about it. That is not ignorance. That is a governance maturity gap, and it is precisely the gap that regulators, and increasingly litigation counsel, are now treating as evidence of negligence.
The World Economic Forum's Advancing Responsible AI Innovation: A Playbook (2025), developed in collaboration with Accenture, identifies unassessed third-party AI tools as a specific barrier to enterprise AI risk management: organisations cannot accurately track AI risks across the enterprise when the tools their vendors are using have never been assessed. The same report finds that less than 1% of organisations have fully operationalised responsible AI in a comprehensive and anticipatory manner.
Regulatory and legal accountability for harm caused by a system it did not build, did not fully understand, and in many cases was not even aware was an AI system. That is the liability-without-visibility problem at the centre of AI vendor risk. It is not hypothetical. It is the scenario the EU AI Act's deployer obligations were written to address.
Your Own AI Governance Framework Sets the Standard for What You Assess
TL;DRYou cannot evaluate a vendor's AI governance against a standard you have not defined for yourself. Before sending any AI risk questionnaire, your organisation needs four things: a complete AI system inventory, a risk classification model, defined human oversight requirements, and clear accountability mapping with named individuals and consequences.
Before you can assess a vendor's AI governance, you need to know what your own framework requires. This sounds obvious. It is also, in my experience, where most organisations are stuck.
They want to send AI risk questions to vendors. They have not yet defined what an acceptable answer looks like for their own organisation. You cannot evaluate a vendor against a standard you have not articulated. The vendor conversation comes second.
Four Things Are Non-Negotiable
First, an AI system inventory: a complete register of AI systems in use across your organisation, including those delivered by vendors. Without it, you cannot even identify which vendor relationships carry AI risk.
Second, a risk classification model. Not all AI risk is equivalent. A chatbot that answers FAQs carries different risk than an AI system making credit decisions or flagging individuals for fraud review. Your assessment questions need to reflect that distinction.
Third, defined human oversight requirements: the floor below which no AI system, internal or vendor-delivered, is permitted to operate without human review.
And fourth, accountability mapping. Named people. Named systems. Named consequences. If you cannot tell me who is accountable in your own organisation when an AI system causes harm, I can tell you with certainty that you are not ready to ask a vendor that question.
Once your internal framework is in place, the vendor assessment question becomes straightforward: do they meet the standard you have set for yourself? Not a higher standard, not a vague notion of 'responsible AI', but the specific, documented standard you have defined for your own organisation. Whisperly's AI Governance Platform is built around exactly this infrastructure: the AI system registry, risk classification methodology, and governance documentation that make the comparison possible, and auditable.
The EU AI Act Has Made AI Vendor Risk a Regulatory Obligation, Not a Choice
TL;DRUnder the EU AI Act, deployers of high-risk AI systems carry regulatory obligations regardless of whether they built the system. Before deployment, organisations must verify conformity, implement human oversight, monitor performance, and report serious incidents. These duties apply to vendor-sourced AI and stack on top of existing sector requirements in financial services, healthcare, and public administration.
The EU AI Act is the piece of legislation that most of my clients are least prepared for in the vendor context. They know the Act exists. Many have assessed their own AI systems against it. What they have not done is work through what it means for their vendor relationships. The answer is uncomfortable: as a deployer, you carry regulatory obligations for high-risk AI systems regardless of whether you built the system. You are accountable for a system you bought, from a vendor you assessed using a questionnaire that did not ask about AI risk. That gap is now a legal exposure.
What the EU AI Act Requires of Deployers in Vendor Relationships
Deployers of high-risk AI systems under the Act must verify that the system was developed in conformity with the Act's requirements before putting it into use. Not after a pilot. Before deployment. They must implement appropriate human oversight, monitor system performance, report serious incidents to the provider and relevant authorities, and maintain operational logs where technically feasible. For deployers subject to regulation in financial services, healthcare, or public administration, these obligations stack on top of existing sector requirements: they do not replace them.
The practical implication for vendor assessment is direct. Before deploying any AI system sourced from a vendor, your organisation must be able to answer the following questions with documented evidence rather than commercial assurances.
| EU AI Act deployer question | What documented evidence the vendor must provide |
|---|---|
| Is this system classified as high-risk under the Act? | The vendor's risk classification assessment, referencing the specific Annex under which any high-risk classification is made. If the vendor asserts the system is not high-risk, the basis for that conclusion must be documented. |
| Has the system undergone conformity assessment? | For high-risk systems, the conformity assessment record or certificate. For general-purpose AI systems with systemic risk, the provider's transparency documentation. The assessment must be current and must cover the version of the system being deployed. |
| What human oversight mechanisms are built in? | A description of the override, monitoring, and intervention mechanisms designed into the system. For systems making consequential decisions, the description must include how a human operator can detect, understand, and correct automated outputs. A statement that the system 'supports human oversight' is not sufficient. |
| How does the vendor handle serious incident reporting? | The vendor's incident reporting procedure, including the timeline for notifying deployers of serious incidents or malfunctions, and the procedure for the deployer to fulfil its own notification obligations to market surveillance authorities. |
| What data was used to train or validate the system? | The training data governance statement: what data was used, from what sources, with what quality controls, and whether personal data was involved and under what legal basis. For EU deployers subject to GDPR, this intersects directly with your data protection obligations. |
One point that consistently surprises clients: the EU AI Act is not limited to EU vendors. Any provider placing an AI system on the EU market, or whose system is used by EU-based deployers, is within scope. This means that a US cloud infrastructure provider, a Singapore-based managed services vendor, or a Canadian SaaS company whose software your EU operations rely on is subject to the Act. Geographic distance does not exempt the vendor, and it does not reduce your deployer obligations. I have had this conversation enough times that it is worth stating plainly.
ISO 42001 and NIST AI RMF: What Responsible AI Governance Looks Like in Practice
TL;DRISO 42001 provides a certifiable management system standard for AI governance; NIST AI RMF offers a voluntary but widely adopted risk management vocabulary. For vendor assessment, ISO 42001 certification gives a verifiable credential, while NIST AI RMF structures the conversation with vendors that have mature but uncertified practices. Using both together covers the widest ground.
Knowing what the EU AI Act requires is necessary. Knowing what it looks like when those requirements are actually met is a different question, and one that the Act itself does not fully answer. This is where technical standards become useful. ISO 42001 and the NIST AI Risk Management Framework do not replace regulatory compliance (nothing does), but they give vendor assessment conversations a vocabulary and a structure that move them beyond the level of vendor representation into the territory of verifiable governance. That distinction matters enormously in practice.
ISO 42001: The Management System Standard for AI Governance
ISO/IEC 42001:2023 establishes requirements for an AI management system: the organisational structures, processes, policies, and controls that govern how an organisation develops, deploys, and monitors AI systems. An ISO 42001 certification from an accredited certification body is the most verifiable signal currently available that a vendor has implemented systematic AI governance rather than informal arrangements. The standard covers risk assessment and treatment, impact assessment for AI systems, controls for data quality and provenance, human oversight mechanisms, and continual improvement processes.
A word of caution about ISO 42001 certification in vendor assessment: the scope matters as much as the certificate. I have seen vendors present an ISO 42001 certificate that covers their internal R&D processes but not the production systems they use to deliver your service. That is not the same thing. Always ask: does the certification scope include the specific AI systems used in the delivery of services to your organisation? If the vendor cannot answer that clearly, treat it as an open question, not a closed one.
NIST AI RMF: The Risk Management Vocabulary for Vendor Conversations
The NIST AI Risk Management Framework (AI RMF 1.0, 2023) organises AI risk management around four core functions: Govern, Map, Measure, and Manage. It does not impose obligations: it is a voluntary framework. It has become the reference point for AI risk conversations in US federal procurement, financial services regulation, and increasingly in cross-border enterprise procurement. Its value in vendor assessment is terminological precision: it provides a shared vocabulary for discussing risk identification, measurement, and mitigation that allows structured comparison across vendors.
| NIST AI RMF function | What it means for the vendor | Assessment question to ask |
|---|---|---|
| Govern | The vendor has defined AI risk management roles, policies, and accountability structures, and has embedded AI risk into its overall enterprise risk framework. | Do you have a defined AI governance policy with named accountability for AI risk at the executive level? How is AI risk reported to leadership? |
| Map | The vendor has identified, classified, and documented the AI systems in its environment, including their context of use, the populations affected, and the potential harms. | Provide your AI system inventory for systems used in the delivery of services to our organisation. For each system, describe the context of use, the affected parties, and the potential harms. |
| Measure | The vendor has established metrics, testing procedures, and evaluation criteria for assessing AI system performance, fairness, and alignment with intended use. | Describe your testing and evaluation regime for AI systems used in service delivery, including how you detect bias, assess output accuracy, and test for adversarial robustness. |
| Manage | The vendor has defined response procedures for AI incidents and performance degradation, including escalation paths, notification procedures, and remediation processes. | Describe your incident response procedure for AI system failures, including the timeline for customer notification and the process for determining root cause. |
The Govern function is where most vendor assessments currently fail. A vendor can have strong encryption, a current SOC 2 report, and a documented incident response procedure, and still have no one who is accountable for what its AI systems actually decide.
Security governance and AI governance are different disciplines. Who is the CISO? Fine. But who is responsible when your vendor's AI system produces a discriminatory output, a hallucinated recommendation, or an automated action that causes harm to your clients? Those are not the same question. The Govern function forces the second one into the open. Most questionnaires never get there.
What AI Vendor Risk Assessment Looks Like in Practice: Moving Beyond the Standard Questionnaire
TL;DREffective AI vendor assessment goes beyond checkbox questionnaires. It requires tiered evaluation calibrated to risk level, live technical validation of vendor claims, contractual provisions covering model changes and incident notification, and ongoing monitoring rather than point-in-time snapshots. The goal is continuous assurance, not a one-off compliance exercise.
The three-layer framework above (internal governance, regulatory compliance, and technical standards) defines what you need to assess. The practical question is how to assess it, given that traditional vendor security questionnaires and due diligence questionnaires were not designed with AI systems in mind. The assessment process needs to be extended in three specific ways.
1. Extend Your Standard Vendor Questionnaire with an AI-Specific Supplement
The starting point is a structured AI supplement to your existing vendor assessment or supplier due diligence process. This supplement should cover:
- 1.AI system inventory for systems used in service delivery
- 2.Risk classification methodology and results
- 3.Training data governance
- 4.Human oversight design
- 5.Incident history and response procedures
- 6.Regulatory compliance status under the EU AI Act and equivalent frameworks
- 7.Certifications held (ISO 42001, SOC 2 AI controls where applicable)
- 8.Model update and versioning procedures
This last point matters because an AI system assessment is not static: a model update can materially change the risk profile of a system without changing its commercial description.
2. Require Evidence, Not Representations
The single most important discipline in AI vendor risk assessment is one I apply in every engagement: ask for evidence, not representations. A vendor that tells you 'we are compliant with the EU AI Act' has told you nothing you can rely on legally or operationally. A vendor that provides a risk classification methodology, a conformity assessment record for the specific system in use, and an ISO 42001 certificate with a scope statement that covers production systems has given you something you can work with. Where certification is not yet held, I find that a specific, dated roadmap with named milestones and a responsible person attached is more useful than any amount of general reassurance. See our supplier due diligence checklist for the evidence framework across the full assessment.
3. Build AI Risk into Your Ongoing Vendor Monitoring, Not Just Onboarding
This is the point where most organisations stop, and it is precisely where the risk continues. A vendor that passed your onboarding assessment in January may have deployed an updated model in March, changed its training data practices in June, and entered a new use case domain by September, none of which triggered a contract review, because the contract did not require notification of AI system changes. I now include explicit AI change notification provisions in every vendor contract I draft: the vendor must alert the client to material changes in AI systems used in service delivery, with a defined timeline and a right to re-assess. Those provisions belong in the data processing agreement and the master services agreement, not left to goodwill or periodic audit cycles that move more slowly than model update schedules.
How Whisperly Supports AI Vendor Risk Assessment and Governance
There is a sequencing problem that comes up repeatedly in AI vendor risk work: organisations want to assess vendors before they have built the internal infrastructure that makes the assessment meaningful. Whisperly's AI Governance Platform addresses that problem directly. The AI system registry, risk assessments, and governance documentation it provides are the prerequisite, not the afterthought. You need to know what your own framework requires before a vendor's answers can tell you anything useful.
For the vendor assessment process itself, Whisperly's vendor assessment and questionnaire automation capabilities allow AI-specific supplements to be structured, issued, and analysed at scale, including matching vendor responses against your internal risk standards and flagging gaps that require human review. For organisations subject to the EU AI Act as deployers, the platform supports the documentation of conformity verification, oversight mechanisms, and incident reporting procedures that the Act requires.
On the vendor side, for organisations that receive AI vendor risk assessments from their own customers, Whisperly's Trust Center allows proactive publication of AI governance documentation: ISO 42001 certification, AI system inventory summaries, risk classification methodology, and EU AI Act compliance status. Buyers who can access this documentation before initiating a formal assessment arrive with many of the most time-consuming questions already answered.
Automate AI Vendor Risk Assessment with Whisperly
Build your AI governance framework, assess vendor AI systems against documented standards, and maintain continuous compliance, all from one platform.
Book a Demo Explore AI Governance
Frequently Asked Questions
What is AI vendor risk and why is it different from standard third-party risk?
AI vendor risk is third-party risk where the source of the risk is an AI system, rather than a human process or a conventional software function. The distinction matters because AI systems fail differently. A conventional system that fails produces an error. An AI system that fails may produce a confident, plausible, and completely wrong output, and the organisation relying on that output may not realise it has failed at all. Security questionnaires designed for conventional software do not ask about hallucination rates, training data provenance, or model drift. They were not built to. The third difference is legal: the EU AI Act places deployer obligations on organisations that use high-risk AI systems from vendors, regardless of whether they built those systems. That makes AI vendor risk a compliance question, not just an operational one.
Does the EU AI Act apply to my organisation if our vendor is not in the EU?
Yes, if your organisation is within the EU or your vendor's AI system is used by EU-based users. The EU AI Act has extraterritorial scope similar to the GDPR: it applies to providers placing AI systems on the EU market and to deployers using AI systems within the EU, regardless of where those parties are established. A vendor headquartered outside the EU that provides AI-enabled services to your EU operations is within scope. Your organisation's deployer obligations apply accordingly.
What is the difference between ISO 42001 and NIST AI RMF for vendor assessment purposes?
ISO 42001 is a certifiable management system standard: it defines requirements that an organisation must meet to obtain third-party certification of its AI governance. NIST AI RMF is a voluntary framework that provides a structured methodology for identifying, assessing, and managing AI risk, but does not lead to certification. In vendor assessment practice, ISO 42001 certification provides a verifiable credential that an independent auditor has assessed the vendor's AI governance against a defined standard. NIST AI RMF provides a conversation framework and assessment vocabulary that is particularly useful when working with vendors who are not yet certified but have mature informal governance practices.
How often should AI vendor risk assessments be conducted?
More frequently than standard vendor assessments, and triggered by events rather than calendar alone. AI systems are updated, retrained, and repurposed at a pace that traditional annual assessment cycles cannot track. At minimum, your vendor contract should require notification of material changes to AI systems used in service delivery, triggering a reassessment of the relevant risk dimensions. Calendar-based reassessment should occur at least annually for high-risk AI systems and at every contract renewal for all vendors deploying AI in service delivery.
What should I do if a vendor cannot answer my AI governance questions?
Treat it as information. A vendor that cannot describe its AI system inventory, explain how it classifies AI risk, or articulate what it does when an AI system fails has not built the governance infrastructure that responsible AI deployment requires. That is a substantive finding, not a questionnaire gap. In practice, I see two types of vendor who cannot answer these questions: those who have genuinely not thought about AI governance as a distinct discipline, and those who have thought about it and are choosing not to be transparent. The right response to the first type is a structured remediation plan with specific milestones and a follow-up assessment. The right response to the second type is a harder conversation about whether the relationship should proceed at all.
Related Whisperly Guides
| Guide | How it connects to AI vendor risk |
|---|---|
| AI Governance Platform | Internal AI governance infrastructure: the registry, risk assessments, and documentation that define your assessment standard |
| EU AI Act Guidebook | Full regulatory framework for AI: deployer obligations, risk classification, conformity assessment, and enforcement |
| ISO 42001 Guidebook | AI management system standard: certification scope, requirements, and how to evaluate a vendor's certification |
| Supplier Due Diligence Checklist | The GDPR and GRC framework for third-party assessment: extends directly to AI vendor risk with the AI-specific supplement approach |
| Vendor Assessment | Whisperly's structured vendor assessment workflow: the platform for issuing and analysing AI-specific questionnaire supplements |
| GDPR Compliance Guide | Data protection obligations that intersect with AI vendor risk, including training data governance and international transfers |
Further Reading on Whisperly
Questions & Answers
What is AI vendor risk and why is it different from standard third-party risk?+
AI vendor risk is third-party risk where the source of risk is an AI system rather than a conventional software function. AI systems fail differently: they may produce confident but wrong outputs. Security questionnaires for conventional software do not ask about hallucination rates, training data provenance, or model drift. The EU AI Act places deployer obligations on organisations using high-risk AI from vendors, making it a compliance question.
Does the EU AI Act apply to my organisation if our vendor is not in the EU?+
Yes, if your organisation is within the EU or your vendor's AI system is used by EU-based users. The EU AI Act has extraterritorial scope similar to the GDPR.
What is the difference between ISO 42001 and NIST AI RMF for vendor assessment purposes?+
ISO 42001 is a certifiable management system standard. NIST AI RMF is a voluntary framework. ISO 42001 certification provides a verifiable credential; NIST AI RMF provides a conversation framework and assessment vocabulary useful for vendors with mature but uncertified governance practices.
How often should AI vendor risk assessments be conducted?+
More frequently than standard vendor assessments, triggered by events, not just calendar. Vendor contracts should require notification of material AI system changes. Calendar-based reassessment should occur at least annually for high-risk AI systems.
What should I do if a vendor cannot answer my AI governance questions?+
Treat it as information, not a gap. The right response depends on whether the vendor has not considered AI governance or is choosing not to be transparent. The first warrants a remediation plan; the second warrants a harder conversation about the relationship.
Reviewed by: Tamara Zavisic, AI Governance Consultant