Most IT managers I talk to are already using AI tools across their organization. Some of those tools were vetted. Many weren’t. The evaluation usually stopped at “does it work” and “what’s the price” — which is understandable given the pace at which these products landed on people’s desks. But the risk profile of an AI tool is categorically different from a traditional SaaS subscription, and the contractual protections that would cover you with a standard cloud vendor often don’t transfer.
This post walks through a practical AI tool risk assessment framework you can apply before signing a new agreement — or use to review agreements you already have in place.
Why AI Tools Require a Different Evaluation
When your company subscribes to a project management platform or a cloud storage service, the vendor’s core obligation is straightforward: store your data, let you access it, don’t lose it. The data doesn’t do anything except sit there waiting for your users.
AI tools are different because the data you put in actively shapes what the system does. Depending on how the vendor has architected their product, your prompts, your documents, your customer records — the inputs your team provides — may be used to train or fine-tune the underlying model. That means your proprietary information could end up improving a product that your competitors also use. It could be retained in ways your data retention policies never anticipated. And if the vendor has a breach, the blast radius isn’t just “someone accessed our files” — it’s “someone accessed a system that learned from our files.”
That’s the threat model you’re assessing against when you do an AI tool risk assessment. It’s worth having that framing clearly in mind before you sit down with a vendor.
Data Handling: The Questions Most People Don’t Ask
Start with the basics, but push past the vendor’s marketing language. “Enterprise-grade security” and “SOC 2 compliant” tell you something about their internal controls, but they don’t tell you what actually happens to the data you submit.
The question you want answered is: where does my data go, and in what form does it persist? Ask the vendor to describe the full data flow for a typical interaction — from the moment a user submits a prompt or uploads a document to the moment a response is returned. You want to know whether inputs are logged, how long they’re retained by default, whether that retention can be shortened contractually, and whether any human reviewers have access to that data for quality assurance purposes.
Human review is a surprisingly common practice among AI vendors. It’s how many of them improve output quality. If your employees are submitting sensitive business information and a vendor employee can read it, that’s a material risk that needs to be disclosed and addressed in the agreement.
Model Training on Customer Data
This is the single issue that creates the most confusion, and it’s the one where vendor language is most likely to obscure what’s actually happening.
The question to ask is direct: does our data train your model, fine-tune your model, or otherwise modify model weights or behavior? Follow up by asking whether that answer changes depending on which pricing tier you’re on. Some vendors disable training on customer data only at enterprise price points. If you’re on a lower tier and haven’t explicitly opted out, you may be opted in by default.
Get the answer in writing, not just in a sales call. What you’re looking for in the contract is an explicit prohibition on using your data for model training purposes, or at minimum, a clear opt-out that’s been confirmed as applied to your account in writing. A vendor that resists putting this in the agreement is telling you something.
If you’re dealing with any regulated data — health information, financial records, data subject to GDPR or CCPA — the stakes here are higher. Many AI vendors are not set up to serve as HIPAA Business Associates or GDPR Data Processors, and signing a standard agreement without addressing this could put you in violation of your own regulatory obligations. The EU AI Act, now in force, adds another layer for organizations with EU exposure, including requirements around high-risk AI system transparency and data governance that don’t disappear just because the vendor is based in the US. You can review the Act’s obligations directly at EUR-Lex.
Subprocessors: The Third Parties You Don’t Know About
AI vendors almost never build every component of their stack themselves. The model might come from a foundation model provider. The infrastructure runs on a cloud platform. Logging and monitoring might route through a third-party service. Each of those entities is a subprocessor — a company that touches your data without having a direct relationship with you.
Ask the vendor for their current subprocessor list. A reputable vendor will have one. Ask how you’ll be notified if that list changes, and what recourse you have if a new subprocessor is added that you consider unacceptable. GDPR requires this kind of disclosure for EU personal data, and the CCPA has similar concepts around service providers and third-party sharing — but even if you’re not squarely in scope for either regulation, the subprocessor list tells you a lot about how a vendor thinks about their obligations to customers.
If a vendor can’t produce a subprocessor list, or the list is vague, that’s a signal about their overall data governance maturity.
Breach Notification: What the Contract Actually Guarantees
Most enterprise contracts include a breach notification clause. The question is whether it’s useful.
A 72-hour notification window is the GDPR standard for notifying regulators after an organization discovers a breach. But many vendor agreements give themselves 30 to 60 days to notify customers after a security incident. That gap matters if you have your own notification obligations running on a tighter clock.
Ask for the specific notification timeline in the agreement, and ask what triggers that clock — some contracts start it when the vendor “determines” a breach occurred, which can be stretched significantly. You want notification of a suspected incident, not just a confirmed one, particularly for AI systems where the scope of what was accessed may not be immediately clear.
Also ask what information you’ll receive in that notification. A notice that says “there was an incident” is not actionable. You need to know what data was involved, what systems were affected, and what the vendor is doing to contain it.
Data Deletion: Getting a Clean Exit
This is consistently the most underspecified area in AI vendor agreements. When you cancel a subscription or terminate a contract, what happens to your data?
For traditional SaaS tools, the answer is usually some version of “we delete it within 30 to 90 days and provide a certificate of destruction if you ask.” For AI tools, the question is more complex because your data may exist in multiple forms: raw inputs in a database, embeddings in a vector store, traces in model weights if training occurred. Vendors who haven’t thought carefully about this will give you a vague answer.
Ask specifically: can you confirm in writing that all forms of our data — including embeddings, cached responses, and any model fine-tuning artifacts — will be deleted upon contract termination? Ask for the timeline and ask for the confirmation method. If the vendor can’t answer this clearly, put a more specific obligation in the contract before you sign.
Contractual Protections: What to Push For
Vendor agreements default to protecting the vendor. That’s not cynical — it’s just true. Your job in the negotiation is to move the contract toward language that actually reflects your risk.
At minimum, an AI vendor agreement should include: an explicit prohibition on training models on your data (or a documented opt-out confirmation), a subprocessor list with change notification, a breach notification timeline that matches your regulatory obligations, specific data deletion obligations covering all data formats, and a clear limitation of liability that doesn’t effectively cap the vendor’s exposure at the monthly subscription fee.
That last point matters. If a vendor’s data handling causes a regulatory fine or a customer breach notification requirement on your end, a $500 monthly subscription cap on damages doesn’t come close to covering it. Push for liability that’s tied to annual contract value at a minimum, and get legal review on anything involving regulated data.
Do This Week
Pull the vendor agreements for your three most actively used AI tools. Open each one and search for these terms: “training,” “model,” “subprocessor,” “notification,” and “deletion.” Document what each agreement says on each point. If a field is blank — meaning the agreement is silent — that’s a gap you need to address either through a contract amendment or a formal email exchange with the vendor that creates a written record.
This exercise takes two to three hours and gives you a clear picture of where your actual exposure is, organized by vendor. It’s also the foundation for any AI governance documentation you’ll need if your organization is ever audited or asked to demonstrate vendor oversight.
If you’re working through multiple vendors at once and want a structured way to score and compare them, the AI Risk Assessor is built for exactly that workflow — it steps through the assessment categories covered here and generates documentation you can keep on file.
The Standard to Hold Vendors To
Reputable AI vendors — the ones who have been in enterprise deals and understand regulated industries — can answer every question in this post. They have subprocessor lists. They have clear training data policies. They have breach notification timelines that map to real regulatory requirements. They’ve had legal review their data deletion obligations.
If a vendor gets defensive or vague when you ask these questions, that’s useful information. It tells you either that they haven’t done the work, or that the honest answer to one of these questions is something they don’t want you to know.
The goal of an AI tool risk assessment isn’t to disqualify every vendor who isn’t perfect. It’s to make sure you’re going in with your eyes open and with contractual protection that actually covers the risk you’re taking on.
