Can You Trust an AI Answering Service to Get It Right?

A caller asks your AI phone agent something it doesn't have an answer for — a niche service question, a policy edge case, a name it's never heard before. What happens next is the whole ballgame. The agent can say it doesn't know. Or it can do what language models do by default: generate something that sounds like an answer, whether or not it's true. On a live call, the caller can't tell the difference until it's too late — they've already believed a false address, a promised callback, or an appointment that was never booked.

Why AI phone agents make things up

Hallucination isn't a rare bug in a phone agent — it's the model working as designed. A large language model doesn't look answers up, it predicts the next most likely words given everything it's seen so far. Ask it something outside what it actually knows, and it will, by default, still predict a plausible-sounding answer, because generating fluent text is what it's built to do. It has no built-in sense of "I don't know."

This isn't specific to voice or to any one vendor. A widely reported 2025 study by the BBC and the European Broadcasting Union tested AI assistants against ordinary factual questions news outlets already had verified answers to, and found a large share of responses contained a significant issue. That's text on a screen, where a reader can pause and double-check. On a phone call there's no screen to check against — a confident voice is the only signal a caller gets, and a confident wrong answer reads as more trustworthy than a hesitant right one.

What we found on one real call

We keep a running incident log for our own voice agent, and the clearest example is worth repeating in full because it shows more than one failure mode in the same call.

A caller asked a question our knowledge base didn't cover. The agent didn't say so — it produced a completed reply with no text in it. Nothing crashed and nothing threw an error, so nothing flagged it as a failure. The caller heard 64 seconds of silence on an open line before hanging up.

Fixing that exposed a worse problem underneath it. Asked where the business was located, the same agent answered the correct city, then appended a street address that exists nowhere — not in the knowledge base, not on the customer's own website. The same call also had it read one toll-free number back three different ways, wrong, across three separate turns. And on a separate real call, an agent told a caller it was transferring him to sales and had booked him a consultation — with no transfer capability on the platform and no calendar connected. He hung up believing he had both. He had neither, and the sale was lost.

What we enforce, and how

The uncomfortable finding underneath all of this: some of those failures happened while a written rule told the model not to. Instructions lower the failure rate. They don't set it to zero.

So we split the problem by where a mistake does the most damage:

• What the agent says in the moment is governed by instructions. It's told plainly that it cannot transfer calls, that when a caller needs a person it should take a message for a callback, and that details like addresses and phone numbers must come from its own sources, not be filled in. Those rules came straight out of the incidents above, including one of our own that used to say "or connect them to someone who can" and was taken literally.
• Silence is handled in code. If a turn finishes without producing anything to say, the agent speaks a fallback instead of leaving the caller on dead air. That check doesn't depend on the model behaving.
• What gets recorded is handled in code. After the call, the outcome a transcript implies is checked against what a tool actually did. A call can't be logged as booked or transferred on the model's word alone — if nothing backs the claim, it's downgraded and flagged for a callback, so nobody on your team skips a follow-up because the dashboard said it was handled.
• Every call is reviewable. The transcript and summary sit next to the outcome, so you can see what was said, not just what the agent concluded.

The last two matter more than they look. A caller told something wrong on one call is a bad call. A dashboard that tells your team "appointment booked" when nothing was booked means nobody calls that customer back at all.

Where voice AI reliability is heading

Latency used to be the industry's whole pitch — shaving milliseconds off the gap between a caller finishing a sentence and the agent replying. That race is largely won; fast responses are close to table stakes across serious vendors now. What's left to compete on is whether the agent is right, and whether it knows the difference between something it knows and something it's guessing. That's a harder problem than latency, because it can't be fixed with faster hardware — it has to be built into what the agent is allowed to say and what the system is allowed to record. The vendors still leading with response time in a year or two will likely be the ones who haven't solved this part yet.

How to test any AI answering service before you trust it with your phone

You don't need to read anyone's engineering docs to check this — including ours. Before you connect any AI phone agent to your business line, call it and try to break it three ways:

1. Ask it something that isn't in its knowledge base — a made-up policy, a service you don't offer, a name it's never heard. An honest agent says it doesn't know and tells you what happens next. One that answers anyway, confidently, is showing you what a hallucination sounds like before a real caller finds out.
2. Ask it to do something it can't do — transfer you, connect you to a person right now. Listen for whether it says so plainly or plays along.
3. Book something, then look at the transcript next to the outcome log. If the two match — the same booking a tool actually created — you're looking at evidence, not a summary the model wrote about itself.

Any vendor should be comfortable running all three tests live, on the phone, before you sign anything. If a sales call gets vague when you ask, that's the answer.