Six questions to ask an AI vendor before you sign

AI vendor demos tend to go well. The product has been rehearsed on the questions it handles best, and the buyer leaves with a picture of what happens on a good day. What matters for a contract is what happens on a bad one. These six questions are meant for the executive across the table, not the engineer.

1. What data does it see, and where does it go?

Ask for a plain account of every place your data travels: which systems it is read from, which provider processes it, where it is stored, and for how long. Ask whether it is used to train anyone's models, and ask for the answer in the contract, not on a web page that can change. If the vendor cannot draw it on a whiteboard, they do not yet know.

2. How did you test it, and can we test it ourselves?

A vendor's accuracy claim describes their test set, not your questions. Ask to run twenty of your own real cases before you sign, including a few you know are awkward. We describe how to build such a set in how to test an AI assistant before it reaches customers. If the vendor resists a trial on your material, treat that as an answer.

3. What does it do when it does not know?

Ask to see it fail. The useful behavior is a clear "I can't answer that, here is a person who can", with the conversation passed along intact. The expensive behavior is a confident wrong answer. Ask how the product decides it is out of its depth, and who gets told.

4. Which risks have you designed for?

The OWASP project publishes a Top 10 for large language model applications that names the common security risks, including prompt injection, where text in a document or message tries to steer the model. You do not need to read it end to end. Ask the vendor which of those risks they have tested for, and what they did about each. NIST's AI Risk Management Framework, a voluntary framework first released in 2023, gives a common vocabulary of Govern, Map, Measure and Manage that a vendor can answer against.

5. Who is accountable when it is wrong?

A product that drafts text for your staff is one thing. A product that talks to your customers, or changes records, is another. Ask what the contract says about errors, who reviews the output, and what logs you get so you can reconstruct what happened. Keep a person responsible for every action that commits the company.

6. How do we leave?

Ask how to export your data, your prompts, your test cases and your settings, in a format another provider could read. Ask what happens to your data after the contract ends, and how fast it is deleted. Models change quickly, so assume you will switch at some point and make that cheap.

A quick filter

If the vendor answers four of these six clearly and in writing, you are in decent shape to run a pilot. If they deflect more than two, wait. If you want a second opinion on an answer you received, we are happy to read it with you. Start at AI development or Nautilus Consulting.