A demo and a promise are not a plan. Here is what directors should ask before they sign off on an AI plan. By Adam Riccoboni.
Most boards will be asked to approve an AI plan in the next year or two, if they have not been already. The paper usually arrives with a slick demonstration, a supplier’s case study and a projected saving.
What it often lacks is the detail a director needs to judge whether the thing will work in this organisation, with these people and this data.
At Critical Future we have spent more than a decade building custom AI systems, and the projects that disappoint rarely fail on the technology. They fail because nobody asked a few plain questions early enough.
None of them requires a director to understand how a model works. They require the same scepticism a good board brings to any capital decision.
1. What decision or process will this change, and how will we know it worked?
“Using AI” is not an objective. A good plan names a specific process, such as answering customer queries, preparing management accounts or scheduling field staff, and says what better looks like: faster, cheaper, fewer errors, happier customers.
It should also say how that is measured today. If there is no baseline, there will be no honest way to tell whether the investment paid off, and the conversation in a year’s time will rest on anecdote.
2. Has it been tested on our data, or only on the supplier’s?
Demonstrations are run on clean, well-behaved examples. Your organisation’s data is not like that. It has missing fields, old codes, spreadsheets that one person understands and exceptions that only turn up at year end.
Ask whether the system has been tried on a real sample of your own work, and what happened on the awkward cases. The gap between the demo and the first month in production is where most of the cost and disappointment sits.
3. Who owns it once it is live?
Every AI system that does real work needs a named person who answers for what it does: Someone who watches its performance, decides when it is retrained or switched off, and is accountable when it gets something wrong.
“The IT team” or “the vendor” is not an answer. If the plan does not name an owner, the board should ask for one before approving it. Accountability is also central to international guidance such as the OECD AI Principles, which New Zealand has endorsed.
4. What happens when it is wrong?
It will be wrong sometimes. The useful questions are how often, how badly and how quickly anyone will notice.
A system that drafts a reply for a person to check carries very different risk from one that sends the reply itself. Ask where the human checkpoints are, what the system does when it is unsure, and how errors are caught and corrected.
A plan that claims the system will not make mistakes is a plan the board should send back.
5. What data does it use, and are we allowed to use it that way?
AI plans often depend on customer, staff or supplier information being used in new ways. In New Zealand the Privacy Act 2020 still applies in full, and the Office of the Privacy Commissioner has published guidance on how its information privacy principles apply to AI tools.
Directors should ask what information goes into the system, where it is stored, whether it leaves the country, and whether people would be surprised to learn how it is being used. Surprise is usually a sign of a problem to come.
6. What is the full cost, including the second year and the way out?
Licence fees are the visible part. The less visible parts are data preparation, integration with existing systems, staff time to check outputs, ongoing monitoring and retraining.
Ask for the cost over three years, not one. Then ask the question suppliers rarely volunteer: If we want to stop or switch, what does that cost, and do we keep our data and the work built on it? Lock-in is far easier to avoid at the start than to escape later.
7. How and when will the board hear about it again?
Approval should not be the last time the board sees the plan. Agree up front what will be reported, how often and against which measures: The baseline from question one, error rates, incidents, cost against budget and any changes in scope.
A short, regular report is worth more than a long launch paper, because it is the only way the board will know whether the promise is being kept.
The answers matter less than the asking
None of these questions is technical, and a board does not need a data scientist in the room to ask them. What it needs is the confidence to treat AI like any other significant investment: clear objectives, honest testing, named accountability and regular scrutiny.
The management team that can answer all seven questions well is usually the one that will deliver. The team that cannot has given the board the most useful information of all.
I have long argued that the real advantage from AI comes from people and machines working together, not from handing the job over and hoping for the best. Boards are part of that partnership. Their questions are what turn a promising demo into something the organisation can rely on.
Adam Riccoboni is the founder and CEO of Critical Future, an AI strategy and data engineering consultancy with over a decade of custom AI development. He wrote The AI Age (2019), co-authored and edited Engineering Mathematics and Artificial Intelligence (Routledge, 2023) and lectures in artificial intelligence at ESCP Business School. This article was written by Adam with the help of an AI writing assistant.
Photo by Sasun Bughdaryan on Unsplash










