AI isn't a technology problem, it's a trust problem

Most of the AI projects I see don't stall because the model can't do the job. They stall at the point where someone has to decide whether to rely on what it produced.

The demo goes well. The pilot goes well. Then the work has to go somewhere real: into production, in front of a customer, into a decision someone will be held to. And the room goes quiet, because nobody is sure they'd stand behind it.

We worked on one project where we found a menial task that was taking each team about an hour a day, and it had a high error rate. We automated it. In testing with subject matter experts, it saved up to 80% of that time.

Then it went to production, and the teams took longer than before. They were double-checking every reference, every link, every individual element of the output.

The system worked the same way it had in testing. The difference was that the people using it now had to put their name to the result, and they didn't believe it yet. What strikes me most is that the manual process it replaced had a high error rate, and nobody had felt the need to check that one line by line.

The technology is there, it has been for a while

The models have been good enough for most enterprise work for some time. Summarising, drafting, classifying, writing and reviewing code, pulling answers out of messy documents: none of that has been the blocker for a while now.

What has changed in the last six months is everything around the model. Connecting it to the systems and data it needs. Giving it the right context. Guardrails on what it can touch. Ways to evaluate whether its output is any good. The model was already capable. It's this scaffolding that turns it into something an organisation can actually get value from, and that's where most of the real work in an AI project goes now.

Some of that scaffolding is also what lets people trust the output: evaluations that show where it's reliable, guardrails that limit what it can get wrong, and logs that show what happened when it does go wrong.

So when an initiative stalls, I've stopped assuming the fix is a better model or a cleverer prompt. Usually the output is fine. The problem is that nobody has a good reason to believe it's fine.

That isn't just my impression. The University of Melbourne and KPMG surveyed more than 48,000 people across 47 countries for their 2025 study on trust in AI. Half of Australians said they use AI regularly. Only 36% said they were willing to trust it.

Why AI is harder to trust than other software

We've trusted software for decades. So what changed?

Traditional software is predictable. The same input gives the same output. When it's wrong, it's usually wrong the same way every time, and you can find the line of code responsible and fix it. Trust came from testing: run it enough times against known cases and you can be confident about the next one.

AI doesn't behave like that. The same question can get two different answers. It can be right 99 times and confidently wrong on the 100th, with nothing in the output to tell you which one you're looking at. When it's wrong, there's often no single line to point at.

None of that makes it untrustworthy. It does mean the ways we've built confidence in software before don't carry over cleanly, and most organisations haven't replaced them with anything.

Trust means different things to different people

When people say "we don't trust it", they rarely mean the same thing.

The executive signing off on a rollout wants to know who's accountable when it goes wrong. The engineer merging AI-written code wants to know it does what it says and won't break something three services away. The frontline team wants to know it won't make them look foolish in front of a customer. The risk and compliance people want to know they can explain a decision after the fact.

Each of those needs different evidence. An accuracy figure on a slide answers none of them particularly well.

What doesn't work

A few things I see organisations try that don't build much trust:

• Quoting accuracy numbers. 94% sounds good until someone asks what happens in the other 6%, and nobody knows.
• Saying "there's a human in the loop" without saying what that human does. If they're approving everything without reading it, all you've added is a signature. In the same study, 66% of employees said they rely on AI output without evaluating its accuracy.
• Banning it. People use it anyway, just without telling anyone, which is the worst of both outcomes. 57% of employees in the study said they'd hidden their use of AI at work.

What does

The organisations I've seen get past this build trust into the system itself. Most of it is unglamorous work.

They start with work where being wrong is cheap and visible, and they measure it. Outputs get compared against what a person would have done, so there's a track record to point to before the system moves on to anything that matters. Trust in a new person on the team works the same way.

They build it so failures are easy to see. Mistakes get logged, flagged, and shown to the people using the system, so they learn where its limits are. Hiding mistakes is the fastest way to make sure nobody trusts the successes either.

They build ownership into the workflow. Every output that matters has a named person attached to it, and the system records who approved what. "The AI did it" is never the answer to a problem, and when everyone knows that, people check their work.

And they let the risk decide how much checking happens. Low-stakes output goes through on automated checks. Higher-stakes output goes to a person, with the sources and reasoning laid out so they can check it properly without redoing the work.

Trust is slow to build and quick to lose

The thing I'd want any leader to understand is that trust in AI is built in small increments and can be lost in one go. One visible failure, in front of the wrong person, can set adoption back months, even if the system has been right thousands of times before.

I don't think the answer is to move slowly. I think it's to be deliberate about where you let it fail, and who's watching when it does.

The title of this post is a bit of a simplification. AI might not be a technology problem, but trust in AI is an engineering problem. It comes from things we already know how to build: evaluation, guardrails, clear ownership, review, and being able to see what went wrong.

The teams on that project checking every reference weren't being difficult. They were showing us what we hadn't built yet.

Part of it was also just time. Trust doesn't arrive on go-live day. People need to watch a system be right, again and again, before they stop checking everything it does, and a rollout plan that doesn't allow for that will look like a failure even when the system works.

So when people won't rely on something we've delivered, I try not to ask why they don't trust it. The more useful question is what we need to engineer so that they can.