How to stop an AI agent inventing prices

You stop an AI agent inventing a price by checking the sentence it wrote before the customer reads it, not by asking it nicely in a system prompt. A model instructed never to guess a number will still produce one, in a tone of total confidence, on roughly the request you would least like it to. The fix is mechanical: compare every number in the draft against the material the agent was actually given, and refuse to send a reply that claims an action the agent has no tool to perform.

Why does a model invent a price at all?

Because a price is the shape of thing that belongs in that sentence, and producing plausible next tokens is the entire job. The model is not lying and it is not confused. "Delivery is usually 3 to 5 working days" is an excellent guess about what a delivery sentence looks like. It is also a commitment your operations team has to keep.

This is why the failure is most likely on the questions that matter most. Price, stock, delivery window and discount are the four fields a customer asks about before buying, and they are the four a model is most fluent at filling in.

What does not work

Telling it not to. Every serious system prompt already says do not invent prices. The thirty-six page blog Apavin had to delete was written under instructions that said do not pad; instructions describe intent, they do not constrain output.

Lowering the temperature. A confident wrong number arrives at any temperature. You get less variety in the guesses, not fewer of them.

Asking the model to check itself. A second pass by the same model, on the same context, agrees with the first one. Self-review catches formatting, not fabrication.

A confidence score. Models are not calibrated on this. The invented delivery window and the real one come back with the same certainty, because to the model they are the same kind of sentence.

All four are worth having. None of them is a control.

The two checks that do work

Both run on the finished draft, after the model is done and before anything is sent.

The first check reads every number in the draft and asks where it came from. Pull the digits out of the reply, then look for each one in the material that produced it: the knowledge base passages retrieved for this message, the catalogue rows, the output of whatever tools ran. A number that appears in none of them was not retrieved. It was written. Small integers get an exemption, because "reply within 24 hours" and "option 2" are not claims about your business, but a price, a stock level, a percentage or a date has to have a source.

This is a text comparison, not a model call. It costs nothing, it cannot itself hallucinate, and it does not have opinions about whether the number seems reasonable.

The second check reads the verbs. The customer-facing agent in Apavin has read-only tools plus exactly two safe writes: remembering a non-sensitive customer preference, and scheduling a reminder the customer asked for. It cannot take a payment, make a booking, issue a refund or change a price, because those sit behind an approval engine with a risk level on every tool and a person on the other side of it.

So a draft that says your booking is confirmed is false by construction. Nothing the agent can do could have confirmed it. That check does not require understanding the sentence, only noticing that it claims a completed action from a system with no way to complete one.

What should happen when a check fires?

The conversation goes to a person, with the reason attached.

Not a retry, because the same context produces the same guess. Not an apology to the customer, because the customer did nothing wrong and has not seen anything yet. Not a silent drop, because then the message is simply unanswered and nobody knows why.

What the customer sees is a short line saying a colleague is picking this up. What the queue sees is the drafted reply, the check that rejected it, and the number or claim that failed. That last part matters more than it sounds: after two weeks the rejected drafts are a list of the questions your knowledge base does not answer, which is the most useful backlog a support team can have.

Does this work in languages other than English?

The number check does, because digits are digits regardless of script, and a customer writing Hinglish in Latin script still gets a price written in Arabic numerals.

The claim check has to be built to. Apavin runs it across languages rather than pattern-matching English phrasing, because an agent that answers in twelve languages and is only guarded in one is guarded in none.

How do you test it?

Write the adversarial cases down and run them on every deploy. Ask for a price that is not in the catalogue. Ask for a bulk discount nobody has authorised. Ask whether an order has shipped. Ask in a second language. Ask the same question after eight turns of unrelated conversation, because context windows fill and grounding weakens as they do.

Apavin's grounding guard ships with a test suite that runs with no API key and no network, so the checks are verified on every build rather than the day someone remembers. A guard that is only exercised in production is a guard you find out about from a customer.

The takeaway

Treat an AI reply the way you would treat a draft from a new hire on their first week: useful, worth reading, and not going out over the company's name without someone checking the numbers. The difference is that with a model you can automate the checking, and the check is cheaper than the apology.