Diligence
Evaluating an AI business in diligence
Diligence on an AI business should test the work the system does when nobody is watching, not the demo. A good review looks at five areas: product reality, data rights, cost to serve, vendor dependence and controls.
Start with the work, not the demo
Demos are built to succeed. A diligence review should ask what the system does on an ordinary Tuesday with ordinary inputs. Ask to watch a live session with a customer's real data, or review a random sample of production outputs that the company did not choose in advance. Compare what the system produced with what a careful person would have produced. The gap, and how the company talks about it, tells you more than any slide.
1. Product reality
- What share of work is completed without a person, and how is that measured?
- What happens to the remainder? Who handles it and how long does it take?
- How was accuracy established: a held-out test set, customer sign-off, or impressions?
- What does the system do when it does not know? Silent guessing is a red flag.
- Which customers use it daily, and which only signed a contract?
2. Data rights and provenance
Models are only as defensible as the data they use. Ask where training and evaluation data came from, what customers agreed to, and whether contracts allow it to improve the system for others. Confirm there is a record of consent and a way to delete data on request. Any personal or regulated data needs a clear legal basis and a clear boundary. Counsel should review this section, not only the engineers. We look more closely at what data can and cannot protect in data moats: real or hype.
3. Cost to serve
Traditional software has near-zero marginal cost. AI products do not. Each request uses compute, and each exception uses people. Ask for cost per unit of work, broken down into inference, human review, support and hosting, and ask how it has moved over the last few quarters. The worked example in unit economics of AI products shows why a healthy-looking price can hide a thin margin.
4. Vendor dependence
Most AI companies rely on outside models. That is fine, provided the company understands the exposure. Ask which providers it uses, what share of cost each represents, and what would happen if one raised prices or changed terms. Ask whether the company has tested a second provider on its own workload, not only on a benchmark. A written switching plan, even a rough one, is a good sign.
5. Controls and governance
Look for the ordinary machinery of a mature company. Is there an audit trail of what the system did and why? Are there limits on what it can do alone? Who approves changes to prompts, models or data sources, and how are changes tested? How are security incidents and model failures reported internally? For regulated customers, ask which obligations flow down to the company and how it meets them.
A short red-flag list
- Accuracy cited without a method.
- Revenue counted from pilots as if it were recurring.
- Cost per unit rising while price stays flat.
- Key behavior living in one person's prompts, with no versioning.
- Customer concentration combined with vendor concentration.
A worked example of a question set
Hypothetical example, for illustration only. A company claims its system resolves most support tickets automatically. A careful reviewer asks for one week of production tickets chosen at random, reads fifty of them alongside the system's answers, and notes how many a trained agent would have handled differently. The reviewer then asks how the company's own accuracy figure was produced, how often the system escalates, and what the customers say in the tickets that came back a second time. If the company's number, the reviewer's sample and the repeat contacts tell the same story, confidence rises. If they do not, the gap is the finding.
People and process
Technology is half of the picture. Meet the people who run the system day to day, not only the founders. Ask what they would fix first, which alerts they ignore and what keeps them awake. Look at how the team decides to ship a change, and whether anyone can stop one. A small team with a disciplined release process is often a safer bet than a larger one that edits live behavior by hand.
Legal and regulatory fit
Depending on the sector, the company may face rules on automated decisions, consumer protection, privacy, intellectual property or sector licensing. Diligence should identify which apply, how the company knows, and who is responsible. This article cannot tell you which rules apply to a given case; counsel should.
What we look for
We prefer companies that can show us their weak spots before we find them. A founder who brings a log of failures, a plan for the next ones and a clear picture of cost per unit is usually a founder we can work with. If you are preparing for a review, our page for founders describes what we ask and how we work.
This article is for general information only and is not investment, legal, tax or accounting advice.
FAQ
Frequently asked questions
How is AI diligence different from software diligence?
Many software questions still apply. The additions are model behavior, data rights, variable inference cost, and the evidence that outputs are reliable at volume.
Can diligence rely on vendor claims?
Treat them as claims. Ask for logs, evaluation results and customer references that show behavior in production.
Who should be involved?
A commercial lead, an engineer who can read the system, and counsel for data and contract questions.
Contact
Building, running or selling an AI-native business?
Tell us a little about it. We aim to reply within two business days.