What Wolt lost when it promoted new restaurants

You open a delivery app on a Friday night, hungry. The easy answer for the app is to put the restaurant you use most near the top.

Favouring familiar choices can also make the marketplace narrower. Familiar restaurants keep receiving orders while new stores struggle to enter a customer’s consideration set. Push discovery too hard, however, and the ranking may serve you worse when you opened the app to repeat last Friday’s order.

Wolt, the Helsinki founded local commerce platform now owned by DoorDash, the US delivery company, turned that tension into an explicit operating choice inside its recommendation system. A recommendation system decides which items to show and in what order. Wolt’s team built one model to rank restaurants, grocers and retailers while tuning how strongly the system favoured first time merchant orders over reorders.

The central question ran deeper than ranking accuracy. Could the Wolt recommendation system create more merchant discovery and hold the platform’s overall conversion rate steady?

A Wolt authored paper released on September 14, 2026 reports three production experiments, which are A/B tests on live traffic, run one after another. The first two covered Wolt’s ten largest operating countries. The third extended the system across every operating country and added retail to a model first used for restaurants.

Wolt had four rankers and one discovery problem

Wolt operates across more than 30 countries and 1,000 cities in Europe and Asia. Unlike a music or video catalogue, its available choices change by delivery address, opening hours, courier capacity, merchant workload and inventory. The list of stores to rank can differ from one request to the next.

The production stack reflected that complexity, with restaurant discovery using three models. A first pass suggested candidates from patterns across similar customers, which marketers know as collaborative filtering. A second pass used context and content. A separate decision tree ranker handled cold start users, meaning customers with no history. Retail used its decision tree model.

Those models also inherited a product conflict. Purchase histories are excellent signals for reordering because they reveal strong habits. The same histories can crowd out new stores. A system optimised only for the next likely purchase may improve short term relevance while concentrating orders among merchants a customer knows well.

Wolt defined three purchase types so the team could inspect the conflict instead of hiding it inside one average. Cold start covered people with no prior history. New covered an order from a merchant the customer was trying for the first time. Recurring covered reorders.

The primary online measures were global conversion rate and merchant trial rate. Merchant trial rate is the share of orders going to a store the customer was trying for the first time. Global conversion blended trial and reorder sessions, while merchant trial rate captured whether the system widened discovery.

One model read order history and another ranked the present

The Universal Venue Ranker, called UVR, combined two different model types. A transformer, a neural network that reads a sequence in both directions, encoded a user’s purchase sequence together with location and time. It produced one score for every candidate store in a single pass.

A gradient boosted decision tree ranker then used that score alongside store content, current context and user and store interaction features. In plain terms, one model learned the shopper’s habits and the second weighed those habits against what was open and available right now. The final ranking therefore combined sequential behaviour with operational information that a purchase history model alone could miss.

This hybrid design was also pragmatic. Wolt ran transformer inference on CPUs and kept end to end p99 latency, the response time that 99% of requests beat, near 60 milliseconds. The figure covers feature retrieval, inference and serialisation. Country level models were retrained daily. Other feature pipelines refreshed from every few hours to once a day.

The team kept the transformer and tree training periods separate to prevent leakage, meaning future information slipping into training. The ranker used 30 days of sessions, with the first 28 days for training, the next day for validation and the final day held out for testing. A two day cutoff stopped the transformer feature from carrying direct knowledge of purchases later used as ranker labels.

Wolt also trained on harder negatives. Instead of comparing the purchased merchant with random stores that sat outside the customer’s view, it used merchants the production system had judged strong enough to display. The ranker had to tell an order apart from plausible alternatives.

Discovery became a dial the team could turn

The first UVR version used a pairwise ranking loss, which is the scoring rule a model learns from. The second changed to a list aware loss and added clicks as a graded relevance signal. Purchases still carried more weight than clicks, and clicks carried more weight than an ignored store.

The decisive control sat at the session level. Wolt assigned greater weight to trial sessions than reorder sessions. Raising that ratio pushed the model toward merchants a customer was yet to try. Lowering it protected familiar ordering behaviour. Think of it as a dial between exploring and repeating.

A 150 run grid sweep, which means trying many dial settings in a systematic pattern, traced the cost with the transformer held fixed. Lower trial weighting improved reorder relevance and could weaken discovery. The highest weighting produced the strongest trial gains at the cost of reorder performance. An intermediate setting improved both more modestly.

Wolt discarded every setting that reduced offline ranking quality for new merchants. From the remaining candidates, it chose the largest trial improvement whose reorder regression stayed within 15%. Germany was the exception, where every tested setting exceeded that bound.

The deployed configuration improved offline trial ranking quality by roughly 12% to 30% over production across six reported countries. Reorder performance declined in five of those six markets under the second version. Germany showed the sharpest cost, with new merchant ranking up 29% and reorder ranking down 27.3%.

The divergence gave the AI ranking experiment a solid reason to go online. An offline aggregate could say little about whether customers would tolerate weaker reorder ranking once the full interface, availability limits and surrounding product logic returned.

Three tests expanded the model one step at a time

All three production experiments used an even 50/50 user level random split covering registered customers, including cold start and returning users. The first two ran for two weeks across the ten largest markets. Those countries represented more than half of Wolt’s global order volume. The third ran for three weeks across all operating countries because the retail test needed a larger sample.

A quick guide to the statistics ahead. A p value shows how likely a result is to be down to chance, and anything below 0.05 is the usual bar for calling it significant.

UVR V1 replaced the earlier restaurant stack. Against the previous production models, it increased merchant trial rate by 5.5% and global conversion by 0.16%. Wolt reports the conversion result with a p value of 0.017. The merchant trial result met its significance threshold of 0.05, although the paper omits the precise p value and confidence interval.

V2 then tested the list aware loss, label smoothing and an extra trial feature against V1 instead of the original baseline. Merchant trial rate increased another 0.45%. Global conversion moved down 0.05%, with a p value of 0.398, so the movement sits within the range of chance.

V3 unified restaurant and retail ranking. It compared the shared system with V2 plus the remaining retail model. Retail merchant trial rate rose 1.31%. Blended global conversion increased 0.04%, with a p value of 0.457. Wolt retired the final legacy retail ranker after the test.

Wolt changed one thing at a time. It first replaced the restaurant stack, then changed the ranking objective, then extended the architecture to retail and every operating country.

The conversion guardrail held, but ad revenue fell

A guardrail metric is a number you watch to make sure a win in one place is not hiding damage somewhere else. The conversion guardrail largely held. V1 improved conversion, while the later movements were small and statistically inconclusive. Offline reorder regressions of more than 20% in some countries stayed invisible in the blended online conversion metric.

The intervention still carried a cost. Advertising revenue declined significantly under V1 and V3. Wolt reports a 0.43% decline for V3. The authors interpret this as the personalised organic ranker outgrowing an advertising stack that still runs on an older ranking model.

The explanation is plausible and remains untested in isolation. The control ranking contained ad placements and deduplication logic beyond the ranker itself. A direct model to model comparison might have produced a smaller gap.

Improving delivery app personalisation changed the inventory and competitive context in which another monetisation system operated. A recommendation upgrade that creates incremental order value can still weaken a neighbouring revenue line, which puts the advertising revenue cost squarely in the decision.

The paper says the three versions produced substantial incremental gross order value, the total value of orders, and leaves out the amount, uncertainty and calculation. Treat that as company attribution, a figure readers cannot independently evaluate.

One model cut complexity, and transfer stays unproven

V3 removed four specialised production rankers and replaced them with one shared transformer plus domain specific tree rankers. The operational benefit is direct. Wolt reduced the number of training and serving systems while keeping country level models and local eligibility rules.

The experiment leaves the reason for the retail gain open. The gain may have come from restaurant behaviour transferring into retail. It may instead reflect retail finally receiving a transformer and tree stack in place of its older tree only model.

Wolt names two missing comparisons that would separate those explanations. One would train the same architecture for retail with the restaurant signal removed. The other would remove the transformer score from an otherwise identical ranker. The team prioritised online rollout within its engineering timeline and says those ablations, tests that remove one component at a time, will follow.

The system also uses purchase histories delayed by as much as one day, leaves out real time clicks and browsing sequences for now, and relies on tree features for stores the transformer has yet to see. These limits make the result difficult to reproduce at a smaller marketplace with less order density or a weaker feature platform.

The ranking objective became a business decision

Wolt converted an ambiguous product goal into an adjustable constraint, measured the damage that adjustment caused offline and tested whether customers noticed it online.

For a European platform spanning markets with different merchant density, customer habits and delivery operations, one global ranking average would have hidden important variation. Wolt trained country level models, reported market level offline conflicts and then tested the product intervention across its largest countries before expanding it globally.

The Wolt recommendation system made discovery measurable through merchant trial rate and made the cost visible through reorder relevance, conversion and ad revenue. After the consolidation, one ranker still had to serve customers, new merchants, familiar merchants, advertisers and the platform at the same time.

Allegro’s five placement recommendation test found the same dependency on context. Its AI recommendations lifted attributed GMV in the cart but produced no measurable gain one step earlier.

More in Growth Signal →

Spotify’s conversational recommendation experiment offers another example of measuring changes in discovery against what users do afterwards.

Experiment detail	What the reporting shows
Change	One Universal Venue Ranker, a transformer plus tree ranker with an adjustable weight favouring first-time merchant orders, tested in three production experiments.
Reported result	Merchant trial rate rose 5.5% and global conversion 0.16% in the first test. Advertising revenue declined, by 0.43% under the final version.
Evidence	Company-authored paper by Wolt and DoorDash researchers, not independently replicated
Limits	Company-authored paper. Sample counts, absolute conversion and order values, confidence intervals and the value of the claimed incremental gross order volume were not disclosed.

FAQ

Sources and limits

• Wolt’s Universal Venue Ranker paper, released September 14, 2026
• Wolt’s official company overview
• Lead author Marcel Kurovski’s LinkedIn profile

Wolt and DoorDash researchers wrote the paper, and independent teams have yet to replicate it. It omits sample counts, absolute conversion and order values, confidence intervals, precise merchant trial p values and the value of the claimed incremental gross order volume. V3 leaves cross domain transfer and the architecture upgrade tangled together, and the advertising revenue explanation is an interpretation that a separate controlled test could confirm or overturn.

Read next

Why AI shopping systems still miss buyer intent Our analysis explores the gap between better recommendations and measurable commercial outcomes.

For another case where more ad engagement failed to produce more revenue, read Douyin’s ad-watching incentive experiment.

Find the reported method, result and limitations of other experiments in the Growth Signal Index in IC Lab.