Written by Gowtham Raj, Director at TartLabs, who leads AI development services and custom software engagements for manufacturing, logistics, and enterprise clients.
The Short Answer
Choose the partner who can answer twelve plain questions about problem definition, data, delivery, risk and ownership with specifics instead of slogans. The research on failed AI projects points to the same place. RAND's interviews with 65 experienced data scientists and engineers most often pointed to organisational causes rather than the model itself, starting with stakeholders who misunderstand or miscommunicate what problem AI is meant to solve. A good partner pushes on that before they quote.
The twelve questions below are the ones we would want a buyer to ask us. Each comes with what a strong answer sounds like and what should make you wary.
Key Takeaways
- RAND's 2024 report cites estimates that more than 80 percent of AI projects fail, twice the failure rate of non-AI IT projects, and names five root causes, most of them organisational.
- Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027 because of escalating costs, unclear business value or inadequate risk controls.
- Gartner also estimates only about 130 of the thousands of agentic AI vendors are real, because many rebrand chatbots and RPA as "agents."
- MIT's NANDA report, as reported by Fortune in August 2025, found purchased or partnered AI tools succeeded about 67% of the time, while internal builds succeeded "only one-third as often" (the report's own wording, on different bases).
- Ask for specifics. A partner who cannot name a metric, a data source and a handover plan has not scoped your project yet.
Every figure above is sourced in the sections below. Our own opinions are labelled as such.
Why the Choice of Partner Matters
Three sources frame the risk, and they measure different things, so read them side by side rather than averaging them.
| Source | Finding | What it measures |
|---|---|---|
| RAND (13 August 2024) | "More than 80 percent of AI projects fail" by some estimates | Interviews with 65 experienced data scientists and engineers, conducted August to December 2023 |
| Gartner (25 June 2025) | Over 40% of agentic AI projects canceled by end of 2027 | A forward prediction, with a January 2025 poll of 3,412 webinar attendees as context |
| MIT NANDA, via Fortune (18 August 2025) | Vendor and partnership builds succeed about 67% of the time, internal builds about one-third as often | 150 leader interviews, 350 employee surveys and 300 public deployments |
Be careful with the first row. RAND's 80 percent is a widely quoted estimate it cites from elsewhere, not a number its own interviews produced. The MIT result comes from a specific 2025 sample, and the report's headline 95% figure drew wide debate, so we use only the partnership comparison, in Fortune's own words, and treat it as directional. We have not converted "one-third as often" into a percentage, because the sentence does not say which base it applies to.
That comparison does not mean any outside partner will do. Gartner's "agent washing" warning says the opposite: many vendors rebrand existing chatbots and RPA without real agentic capability, and the firm estimates only about 130 of the thousands of agentic AI vendors are genuine. Your job is to tell the real ones apart, and the questions below are built for that.
Questions About the Problem and the Plan
1. "What problem are we solving, and how will we measure success?" This comes first because RAND found misunderstood or miscommunicated problems were the most common cause of failure, with models "optimized for the wrong metrics." A strong partner writes the success metric down before the contract, in business terms such as handling time, error rate or revenue per account. Be wary of anyone who answers with a technology name.
2. "Is AI the right tool here at all?" Gartner notes that "many use cases positioned as agentic today don't require agentic implementations." A partner who sometimes recommends a rules engine or a plain workflow tool over a model is a partner optimising for your outcome. We treat it as a warning sign when a partner has never talked a client out of AI.
3. "How will you test the idea before we commit to the full build?" Look for a short, paid pilot with a go or no-go decision at the end. RAND advises committing a team to a problem for at least a year, so the pilot should test whether the problem is worth that commitment, not just whether a demo can be built.
4. "Who on your side talks to our business people, and how often?" RAND's first recommendation is that technical staff understand the project purpose and domain context. Ask for the named person, the meeting cadence and a sample status report. If the answer is "through the project manager," ask again.
Questions About Data and Technology
5. "What data do we need, and what shape is ours in?" RAND's second root cause is an organisation lacking the data to train an effective model. A good partner asks to see your data in the first week and tells you honestly what is missing. Beware the partner who quotes a fixed price before looking at a single record.
6. "Which models and tools will you use, and why those?" RAND lists chasing the latest technology instead of solving real problems as a failure cause. A strong answer ties each choice to your constraints: cost per request, latency, data residency and the ability to swap a model later. Ask whether you can change providers without a rewrite.
7. "How does this connect to our existing systems?" Gartner warns that integrating agents into legacy systems "can be technically complex, often disrupting workflows and requiring costly modifications." Ask for an integration plan naming the systems, the interfaces and who owns each. For older estates, see our guide to integrating existing systems with new software.
8. "What infrastructure will this need in production, and who pays for it?" RAND's fourth root cause is inadequate infrastructure for managing data and deploying models. Ask for a run-cost estimate: hosting, model usage fees, monitoring and retraining. Gartner cites escalating cost as a reason projects get canceled, so a partner who gives you only a build price is hiding half the bill.
Questions About Delivery and Risk
9. "Who exactly will work on this, and what happens if they leave?" Ask for the named team, their seniority and their tenure with the firm. A dedicated team model can give you continuity, but only if the contract says who stays. Make sure the people in the sales meeting are the people who build.
10. "How do you handle security, privacy and wrong answers?" Gartner lists inadequate risk controls among the reasons for cancellation. Ask how the system behaves when the model is wrong: human review, confidence thresholds, audit logs, and where your data goes during processing. A weak answer is "the model is very accurate." A good answer describes what happens on the one request in a hundred that it gets wrong.
11. "Show me a similar project that reached production, and let me speak to the client." In our view, the best reference is a project that has been live for at least a year, so ask for its uptime and incident history. Then ask what the project cost against what was quoted, how long it took, and what broke. Treat a refusal to provide a reference as an answer in itself.
Question About Ownership
12. "Who owns the code, the data and the trained assets, and what does handover look like?" Get this in writing: source code, prompts, fine-tuned models, evaluation sets and documentation should all be yours, with a handover plan that does not depend on the partner staying involved. It also keeps your options open if you later bring work in-house or change vendor, which the MIT comparison suggests is worth deciding deliberately rather than by default.
How to Use the Twelve Questions
- Send them in writing before the first call. Written answers are easier to compare, and vague ones stand out.
- Score each answer from 0 to 2: 0 for a slogan, 1 for a general answer, 2 for specifics with a number, a name or an example. Totals matter less than the pattern of zeros.
- Weight the first four questions double. They cover problem definition and the plan, which RAND lists first among its root causes and describes as the most common reason for failure.
- Run a small paid pilot with your top two. Real work reveals more than any proposal. Our AI strategy guide for software companies covers how to frame that pilot and where it fits in a wider plan.
- Check your own readiness too. A partner cannot fix missing data access or an absent business owner for you.
The Bottom Line
Picking an AI partner depends on how well they understand your problem, more than on which model they favour. RAND's interviewees named misunderstood purpose, missing data and weak infrastructure among the leading causes of failure, and Gartner cites escalating cost, unclear value and inadequate risk controls as reasons for cancellation.
Use the twelve questions to make a vendor show their thinking before you pay for it, run a small pilot with your shortlist, and put ownership and handover in writing. If you would like a second set of eyes on a proposal you have received, or want to see how we would answer these questions for your project, get in touch or read our AI strategy guide.
