8 min read
Editor’s note The main study was written by employees of companies selling a verification product based on its findings. Its sample, comparison and commercial interest are stated alongside the results.
TL;DR
- Across 90 commerce sessions, ungrounded agents claimed progress unsupported by event logs at rates from 2.4% to 8.5%, depending on the model.
- Supplying verified transaction state reduced unsupported-positive rates to between zero and 0.4%. Telling models to be cautious mainly made them refuse to answer.
- Merchants should make event logs authoritative, verify high-risk claims before delivery and test the same scenario repeatedly. A single successful demonstration does not establish reliability.
The failure in one sentence
An AI commerce agent can say an order succeeded when the transaction log shows that no order exists. Researchers from Rezolve AI Labs and the University of Sarajevo call this a journey hallucination, a mismatch between what the agent says happened and what the system of record shows.
“The cart is currently empty, the checkout process has started, and an order has been completed.”
That response came from the study’s user test. Two of its three statements were false. Transactional errors are different from an invented statistic because other systems act on them. A false order can trigger confirmation messages, delivery expectations, support contacts or refund requests for a purchase that never happened.
The practical fix is architectural. The model should not infer transaction state from conversation history when the cart, checkout and payment systems already record what happened.
What the study found
The UMAP paper examined 90 real ecommerce sessions containing about 2,000 events from two merchants on one conversational-commerce platform. Sessions averaged 22 events and all included at least one cart action.
Researchers replayed each session step by step, asked GPT-4o, DeepSeek-V3, Kimi-K2-Instruct and Qwen3-32B to state the transaction status, and compared each claim with the event log. The most important measure was the unsupported-positive rate, the share of answers claiming progress that the log did not support.
| Model | Unsupported positives without verified state | With verified state | Ungrounded coverage |
|---|---|---|---|
| GPT-4o | 8.5% | 0.4% | 62.3% |
| Kimi-K2-Instruct | 4.1% | 0.2% | 99.4% |
| Qwen3-32B | 3.1% | 0.0% | 63.9% |
| DeepSeek-V3 | 2.4% | 0.0% | 81.2% |
GPT-4o’s 8.5% rate is roughly one unsupported claim in twelve measured interactions. The result should not be treated as a universal production benchmark. It shows the order of magnitude in one small dataset and why every merchant needs to measure its traffic.
A common fallback is to tell the model to admit uncertainty. The researchers tested that too. Qwen declined roughly 85% of questions. Kimi answered about a quarter, while its unsupported-positive rate increased from 4.1% to 8.1%. Caution mostly reduced usefulness through abstention.
Three different failures
| Failure | What goes wrong | Business consequence | Right measure |
|---|---|---|---|
| Factual hallucination | The agent invents a product fact, policy or citation | Bad advice or misleading information | Claim accuracy against a trusted source |
| Journey hallucination | The agent asserts a cart, checkout or order event absent from the log | Incorrect automation, customer confusion and support cost | Unsupported-positive rate by event type |
| Repeated-run failure | The agent succeeds once but cannot repeat the same task reliably | A pilot overstates production performance | All-runs pass rate across repeated attempts |
The distinction is necessary because one overall accuracy score can hide the errors that cause the most damage. Coverage, unsupported positives, missed real events and false order-completion claims should be reported separately.
The highest-severity claim is order completion. A merchant can tolerate an agent asking for clarification more easily than an agent telling a customer that a nonexistent payment and order succeeded.
Why one successful run proves little
Independent work points to the same production problem from another direction. On τ-bench, a GPT-4o retail agent scoring above 60% on one attempt fell below 25% when success was required across eight runs.
The measure, often called pass^k, asks whether the same task succeeds every time across repeated attempts. A commerce agent does not handle one customer once. It repeats similar tasks at volume, so the all-runs result is closer to the operating reality than the best demonstration.
This does not mean the UMAP study and τ-bench measured the same error. One compared transaction claims with event logs. The other tested whether agents completed tasks consistently. Together they show why grounding and repeated testing solve different parts of the reliability problem.
Five requirements before launch
- Define the dangerous claims. List the statements that can trigger customer action, cart contents, checkout status, payment success, order creation, cancellation and refund status.
- Make the event log authoritative. Reconstruct state from cart, payment and order events rather than asking the model to infer it from conversation history.
- Verify before delivery. Check structured claims in the serving path. A nightly report documents the error after the customer has already read it.
- Separate error types. Report coverage, unsupported positives, missed real events and order-completion violations independently.
- Repeat realistic scenarios. Run empty carts, add-then-remove sessions, abandoned checkout, completed orders followed by new browsing and multiple simultaneous errors several times each.
A useful initial audit can start with fifty recent sessions in which the assistant said something about a cart, order or delivery. Compare the response with the underlying events, calculate the unsupported-positive rate and identify which claim types created the exposure.
Commercial verification products can implement the pattern, but buying one does not remove the need to understand it. Ask vendors for the unsupported-positive rate, the comparison baseline, whether verification runs before the response, what happens when a check fails and which events their method requires.
What the evidence cannot establish
- The UMAP paper is a short Late Breaking Results paper using 90 sessions from two merchants.
- Its authors include employees of Rezolve Ai, which sells a verification product based on the approach.
- The comparison tested prompt configurations, not competing production middleware or session-memory products.
- The user study involved 42 people recruited from organisational and personal networks.
- The method catches claims that contradict explicit traces. It cannot detect every misleading response.
One result complicates the commercial story. In a scenario containing a subtle checkout error, 71% of participants accepted the inaccurate response while only 29% accepted the verified one. Presentation influenced trust even when verification improved correctness. A technically accurate state still has to be explained clearly.
The evidence therefore supports a narrower conclusion than “verification solves hallucination.” Event grounding sharply reduced specific false claims in this dataset. It did not establish universal reliability, a superior commercial product or trustworthy communication alone.
Atlas24h’s transaction verification provides a practical counterpart: the operating system confirms a contract rather than accepting the model’s account of what happened.
Sources and method
| Source | Used for | Independence |
|---|---|---|
| UMAP study | Journey-hallucination concept, model figures and user study | Peer-reviewed short paper; authors employed by a vendor selling the approach |
| τ-bench | Repeated-run reliability and pass^k | Researchers affiliated with Sierra, which sells AI agents; method is public |
| Air Canada case | Example of business responsibility for chatbot information | Public tribunal decision and legal reporting |
This article contains no first-party data. External figures retain their source, sample and limitation so readers can distinguish a measured result from a general production claim.
FAQ
What is a journey hallucination?
A journey hallucination occurs when an agent reports a cart, checkout or order event that the execution log does not support. Unlike an ordinary factual error, the claim can trigger automated commercial workflows.
How often do agents get transaction state wrong?
In one 90-session UMAP study, unsupported-positive rates ranged from 2.4% to 8.5% under ungrounded prompting. The sample is too small to serve as a universal benchmark, so merchants should measure their sessions.
Can cautious prompting solve the problem?
Not reliably. In the study, cautious prompting mostly increased refusal rates and, for one model, increased unsupported positives among the answers it still produced.
Does a merchant need a commercial verification product?
Not necessarily. The underlying control reconstructs transaction state from existing events, supplies verified state to the agent and checks high-risk claims before delivery.
Why test the same task repeatedly?
Single-run success hides inconsistency. Repeated-run testing measures whether the same scenario works reliably enough for production volume.
Related reading
- Agentic checkout, on why discovery inside assistants is outrunning the buy button.
- AI product ratings, on what the model reads before recommending a product.
- Agent advertising, on measuring machine-heavy traffic.
Spotted an error? Contact the desk. Our corrections policy explains how we handle corrections and updates.