Your customers were meant to trust every answer.
AI where judgement lives, code where rules live, people where accountability lives.
Three short opinions on what makes an answer worth trusting.
Use AI where it's needed. Use code where it isn't.
Most of a business process should not be decided by a language model. If a rule can be written down, write it in code. Code is cheaper, faster, testable and gives the same answer every time.
Take an invoice. Reading a supplier's PDF and working out which line is the freight charge is a job for AI: the layouts vary and the wording is messy. Checking that the invoice total matches the purchase order within 2% is not. Neither is deciding that payments over $10,000 need a controller's approval. Those are rules. Put them in the model and you get a system that is right most of the time, which for payments means wrong often enough to matter.
So we draw the line deliberately in every workflow. AI reads, classifies, drafts and suggests: the parts that need judgement over unstructured input. Code checks, calculates, enforces limits and decides what needs a person. The model proposes; the code decides whether the proposal is allowed.
This has three practical effects. First, cost: a model call for a decision that an if-statement could make is waste, and it adds up across thousands of transactions. Second, testing: code paths are tested the usual way, and only the AI steps need evaluation sets, which keeps those sets focused. Third, trust: when an auditor asks why a payment was approved, the answer is a rule and a name, not a probability.
The opposite mistake exists too. Some teams avoid AI where it clearly helps, and end up with brittle templates that break on the next supplier. The point is not less AI. It is AI in the right places, with code around it.
AI or code?
AI, twice: to build it and inside it
We use AI in two places: to build your software, and inside the software we build. They need different disciplines, and confusing them causes most of the problems we see.
Building with AI. Our engineers use AI to write code, tests and migrations. It makes them faster, but speed is not the goal. Working software is. So AI-written code runs inside a harness: a written spec with examples, tests derived from those examples, type checks, and review by a senior AI engineer. Code that fails the harness does not merge. The model can write the first draft of a function; it cannot decide the function is done.
Building AI in. Inside your product or operations, AI does real work: reading documents, answering customers, preparing decisions. Here the discipline is different. The output cannot be checked by a compiler, so each AI step gets an evaluation set of real cases, a threshold per category, and monitoring after launch. Rules the business depends on stay in code. A person approves what matters, and every action is logged.
Why both? Because the second depends on the first. An agent is only as reliable as the software around it: the permissions, the records it reads, the checks on what it writes. Teams that build the agent carefully and the surrounding system hastily end up with an impressive demo on a weak base.
And because it compounds. Every rule, connector, evaluation set and guardrail we build is reused. Your project starts with what we've already solved, and the AI we use to build it works faster on a base it has seen before.
Why AI pilots stall before production (and how to get past it)
Most AI pilots stall for the same reason: they prove the model can do the task, but not that the business can rely on it. A demo on ten hand-picked examples says nothing about the eleventh case, the one with a missing field and an angry customer.
We see four gaps again and again.
No definition of good. Nobody agreed what accuracy is acceptable, per category, before building. So every error becomes a debate. Fix: write the evaluation set and thresholds first, with the people who own the process. Some categories need 100%.
No connection to real systems. The pilot reads exports and writes to a spreadsheet. Production means real permissions, real records and real side effects. Fix: connect to the actual ERP or EHR in a sandbox from week one, under a limited role.
No place for people. Either the agent does everything or nothing. Fix: design the approval steps up front. Let the agent handle clear cases by rule and route the rest to a named person, with the context prepared. Start with a high handoff rate and lower it as evidence builds.
No owner after the demo. The team that built it moves on, and nobody watches quality, cost or drift. Fix: monitoring and runbooks are part of the scope, and someone stays until it works in production.
None of this is about a better model. The model is usually good enough. What is missing is the ordinary engineering around it: tests, permissions, approvals, logs and ownership. That work is less exciting than the demo, and it is the part that decides whether the pilot ever pays off.
Which moment would you fix first?
Tell us where your customers wait today. Our AI assistant, Ask Unthinkable, helps you define your requirements, then connects you with the right team.
We use essential cookies to run this site. With your consent we also use analytics cookies to see which pages help. Privacy policy