Industry
Contents
Business and AI, with a European lens

Growth Signal Index

The Growth Signal Audit of 51 Business Experiments

11 November 2026 snapshot: Get the CSV · Read the dated methodology. Underlying source-check dates are recorded separately from the snapshot publication date.

Live index. Updated on every new experiment.

Explore what businesses tested, the results they reported and the evidence behind them.

Filter the tests, then open a row for sources and method. “Controlled” means the test used a control group.

57 growth tests 36 with a control 44 report a gain 11 mixed or inconclusive Latest added 20 Nov 2026
Read Growth Signal

Company Growth bottleneck Change tested Reported result Growth hack Details
Other Industry Contents research
Test whether a first cold email that leads with a performance statistic or one that leads with the reader's problem earns more attention from a data-trained audience. Randomise 2,247 researchers between problem-led and statistics-led first emails, holding sender, personalisation, length, validation link and send window constant. 49.6% vs 38.8% open rate Controlled In a first cold email, name the reader's problem and keep the proof one click away.
Travel and events Airbnb
Guest embeddings could lag recent browsing and searches by almost two days. Event-triggered inference refreshed the existing Transformer embedding in 10 to 30 seconds. ~0.3% No control shown Keep the model fixed and randomise only feature freshness, then judge the stream on a downstream commercial metric and infrastructure guardrails.
Retail and ecommerce Wine Access
Test whether AI could replace or assist three salaried newsletter writers without reducing realised orders and gross profit. Randomise 23,970 subscribers between human, Claude and Claude-plus-marketer copy while holding the wines and inputs fixed, then apply a net-profit decision rule. 23,970 subscribers randomly assigned Controlled Hold the offer fixed, randomise the production workflow, and decide on realised gross profit plus operating cost with brand and accuracy guardrails.
Media, social and ads Netflix
Personalise new artwork before sparse canvases and newly launched assets accumulate enough interaction history. A unified model pooled interaction data across five artwork canvases while CLIP embeddings represented what each image contained. Only the combined model won the online A/B test Controlled Test the AI feature, its supporting data change and their combination separately before concluding that the feature itself failed.
Retail and ecommerce GARCIA
Shoppers still could not judge how garments would look after receiving size advice. AI virtual try-on added above the existing size-me tool in the treatment arm. 7.06% No control shown Test visual confidence separately from size advice and decide on contribution margin after returns mature.
Retail and ecommerce Kruidvat, AS Watson Benelux
Large baskets pushed the main checkout button below the fold while a two-person CRO team managed more than 100 experiments a year. Use Contentsquare insight and Kameleoon PBX agents to generate and configure a sticky checkout control, then require human QA before launch. 30 minutes from insight to launch-ready test Controlled Automate bounded build and setup tasks, keep human QA, and let a controlled business metric decide rollout.
Retail and ecommerce eBag
Help repeat grocery shoppers complete a long basket without relying on a new visit or discount. Shopper-level recommendations used live catalogue data and order history, including a cart module for previously bought items missing from the current basket. 30% of 693,666 orders contained a clicked recommendation No control shown Use past purchase history in one bounded cart-page module, then reserve a persistent holdout and judge incremental revenue or margin per eligible visitor.
Retail and ecommerce ASOS
Make high-intent search results more relevant without replacing the shopper's query or rebuilding the catalogue. ASOS brought its recommendation engine into search so recent views, purchases, bag additions and saved items could rerank results for logged-in shoppers. 140 basis point reported conversion lift No control shown Test recommendation signals as a bounded reranking layer inside search, keep a persistent holdout and annualise only after publishing the eligible audience and test window.
Retail and ecommerce Smartphonehoesjes.nl
Experiment building and analysis constrained the CRO team’s testing capacity across three European storefronts. Claude Code built most variants, while n8n and Airtable prepared country-level data for an AI proponent, opponent and jury governed by predefined business rules. 90 experiments in five months No control shown Automate low-risk building and first-pass analysis, but route conflicting and delayed business metrics to a human reviewer.
Telecom and services Allianz Benelux
Allianz needed to reduce avoidable churn among car insurance customers renewing after a price increase. A randomised 90/10 test compared the standard renewal email with a policy-data-driven personalised video journey built through Lont's governed AI-enabled workflow. 10.9% relative churn reduction over 100 days Controlled Bound AI inside approved content and deterministic rules, keep a live holdout on the incumbent journey and judge the system on retention rather than engagement.
Retail and ecommerce Sweaty Betty
Sweaty Betty's syndicated product feed described products in less depth than its website, while manual product-level highlight rules did not scale. Matched yoga and running product sets compared the normal feed with AI-generated Google Shopping product highlights. 80% reported yoga revenue gap versus control Controlled Treat AI enrichment as a product-data intervention, preserve matched item-level controls, and follow the result from impressions through revenue.
Retail and ecommerce Paulmann Licht
Paulmann needed to reduce empty search results and manual relevance tuning while increasing revenue from a complex product catalogue. A six-week 50/50 bake-off compared its incumbent search with Dynamic Yield Experience Search, initially in default configuration. 26% reported revenue uplift among search users Controlled Test an AI search system in its default state against the incumbent before tuning, and judge the rollout on revenue with a persistent holdout.
Retail and ecommerce Karaca International
Karaca wanted selected German product pages to convert more visitors without changing traffic acquisition. Split traffic evenly between existing descriptions and Cypien-generated German copy for a Home Enthusiasts segment across 50 products. 7% relative conversion lift versus control Controlled Use AI as a cheap variant factory and let a randomized holdout decide whether the content scales.
Retail and ecommerce BOXIO
BOXIO wanted faster Shopify navigation without rebuilding its store and required proof from its own order data. Split traffic evenly between the original site and Speed Kit's caching, service-worker and predictive preloading layer. 9.65% higher user conversion versus control Controlled Require an acceleration layer to win a random traffic holdout on orders before full rollout.
Other Fischer Future Heat
Around 2,000 enquiries a week remained unbooked after seven days of sales follow-up. Route only day-seven leads to an AI SMS agent that asks three qualification questions and books eligible homeowners into the survey calendar. 105 qualified home surveys booked in 30 days No control shown Put automation after a clear human follow-up boundary, check the texts against PECR, and test closed revenue against a random holdout.
Travel and events Javo
Growth teams need to know whether an AI sales agent adds revenue across all visitors, including the many who never open the chat. A visitor-randomised design compared no agent, Javo and a frontier-model agent inside the same interface, retrieval layer, tools, prompt and guardrails. 10% higher revenue per visitor Controlled Keep the interface, tools, prompt, guardrails and measurement fixed so a growth test changes one part of the AI selling system.
Travel and events Spa Grand Prix
Fans picked seats, budgets and extras across many languages, and each choice depended on the last. Added Jack, a voice and text AI agent that answers questions, shows 360 degree seat views and fills the basket. 70% higher checkout clickthrough, €539,000 extra revenue anticipated No control shown Measure paid orders against a holdout before counting anticipated revenue.
Travel and events Radisson Hotel Group
Travellers shortlist hotels in AI assistants first. Could that discovery lead to direct bookings? Connected ChatGPT hotel discovery to live property data and a handoff to the Radisson booking site. Bookings converted at about 1.5 times the organic search rate No control shown Compare matched audiences before calling a channel gap incremental.
Telecom and services Atlas24h
A chatbot answered product questions but captured no leads and made no sales. Let eligible visitors buy inside the chat, with structured lead handoff to sales for the rest. €113,722 attributed revenue from 1,705 contracts in July No control shown Verify each claimed sale in the operating system, then hold back a slice of traffic to see how many sales would have happened anyway.
Software and AI Algolia
A pricing page test needed custom code, and the small development team had other priorities. Used an AI agent to build and configure two live variants, with people keeping review and launch control. 15% lift in an unnamed metric No control shown Use AI to speed up variant building, and keep hypothesis, metric and launch decisions with people.
Media, social and ads Spotify
A multi turn recommendation agent needed testing before real multi turn data existed. Generated replayable synthetic dialogues, audited them with human annotators and used failure traces to improve the agent. Weekly active users up 5% in a two week test Controlled Use replayable dialogues and a small human audit to find failures before paying for a live test.
Media, social and ads Teads
Short, fragmented user histories made sequential targeting hard inside strict bidding time limits. Fed short event sequences into a reusable user model, tuned by a bounded set of LLM proposed changes. Click through rate up 2.13% in a seven day test Controlled Let AI propose changes only from an approved list, and require a live control before claiming growth.
Software and AI TreeHouse
Manual prompt checks could not link AI search visibility to paid trials across four rental sites. Linked automated prompt monitoring and crawler logs to referral tracking for paid trials. AI search drives about 25% of organic conversions No control shown Tie AI referrals to a commercial event before optimising citations.
Software and AI Zero
Pipeline records went stale because updating them by hand took time, so teams stopped trusting the data. AI agents read email, calls, usage and billing, then update CRM records. Only outbound messages wait for approval. Six customer stories, no published error rate No control shown Time how long it takes to correct the AI written records during a trial before trusting the savings.
Telecom and services Sterling Insurance Services
Shoppers the broker paid to attract hesitated over cover or wording and left before buying. An AI agent predicted likely exits and answered questions from the live policy wording, starting with two learner driver schemes. Quote conversion up 31.4% relative over eight months Controlled Start with one product family whose documents agree, keep a control and watch how the rest of the journey changes.
Retail and ecommerce RS Group
Reject all cookie consent left paid search bidding learning from incomplete customer journeys. Rebuilt journeys from first party event data and fed the conversion signal into automated bidding in a four month Italy pilot. 6% more revenue on 5% less spend No control shown Treat the conversion feed as a growth lever and test it against a stable holdout before scaling.
Retail and ecommerce Galeria
Replacing site search with a usage priced AI system meant sales had to outgrow a new variable cost. Cleaned the product feed, then A/B tested AI commerce search across 1.5 million sessions. 15% sales lift across 1.5M sessions Controlled Judge a metered tool on revenue in the same unit as the bill.
Retail and ecommerce The Very Group
Fixed triggers showed the same offers to shoppers with very different intent. An AI agent picked which approved experience each shopper saw, using live intent signals. 20% more revenue per customer in the first month No control shown Test the agent against your existing rule and judge it on margin as well as revenue.
Telecom and services Telefónica Deutschland (o2)
The most valuable visitors were anonymous, so profile based rules could not follow a changing session. Joined live session behaviour with product feed data and AI recommendations to refresh home page hero banners. Conversion up 275% relative on personalised banners No control shown Validate one narrow placement first and keep a holdout on the old rule.
Travel and events Robinson Club (TUI Group)
Landing pages and competitor research took one or two days, slowing the start of each test. Taught an AI the brand voice, page frameworks and CMS components, then added agents to check its output. Page build time cut from days to seconds No control shown Give AI only the page parts and checks you already trust, then time each test from request to launch.
Retail and ecommerce marko.pl
AI recommendations had to beat a tuned manual setup, and early click gains did not match conversion. Ran five A/B tests on widget placement, section order and price similarity. Recommendation purchases up 115%, overall conversion up about 4% Controlled Set the deciding metric before launch and check whether recommendations took sales from elsewhere. More clicks can hide flat sales.
Software and AI Fyxer
Building an experiment took days or weeks, limiting how many tests ran. Built a reusable AI workflow linking shared instructions, coding agents, internal data and the experiment API. 360 tests by four engineers in a year No control shown Find the slowest repeatable step between idea and decision and hand AI one narrow job there.
Retail and ecommerce Zalando
Test and control groups competed for the same advertiser budgets. Split each campaign budget into virtual sub campaigns before the auction, so simultaneous tests can share campaigns. Experiments up from 8 to 60 plus a year No control shown List what test and control share, such as budget, and split it before the test starts.
Retail and ecommerce Deliveroo
A high recall model raised more churn alerts than the retention team could act on. Ranked restaurants by churn risk, set a threshold to match team capacity and planned a randomised holdout. Alerts down over 90% offline, precision up eightfold No control shown Send only as many alerts as the team can act on, and keep a comparable group untouched to learn whether outreach keeps customers.
Retail and ecommerce Matas Group
The loyalty team could not tell which dormant subscribers to target, or prove that re engagement changed retention. Used explainable AI to find a dormant cohort and withheld the re engagement flow from a control group. Retention up 29 percentage points after 30 days Controlled Keep audience selection separate from treatment assignment and preserve a control to measure incremental retention.
Telecom and services Orange France
A universal €10 offer leaked discounts to shoppers who would have bought anyway. A model showed the offer only to hesitant visitors, tested against no offer and the universal offer. Promo code use down from 68% to 25% of purchases Controlled Test a targeted discount against both no offer and a blanket offer, and judge it on margin, not conversion alone.
Retail and ecommerce Wolt (DoorDash)
Purchase history favours reorders and crowds out new merchants. Added an adjustable weight to a venue ranker that favours first time merchant orders. Merchant trial rate up 5.5%, ad revenue down Controlled Make discovery an explicit setting, then check what it costs reorders and neighbouring revenue such as advertising.
Retail and ecommerce Allegro
Co purchase data confused complementary products with alternatives, and results varied by placement. Tested category guided recommendations on product page, pre cart and cart placements. Cart GMV up 21.25% on web, up 15.73% in app Controlled Test recommendations in each placement, because a model that lifts cart sales can fail one step earlier.
Software and AI SignalTo.ai
A new ad platform had no public benchmarks and two offers to compare. Ran a question led ad and an offer led ad in ChatGPT, with context hints for targeting. 7 report requests, all from one ad No control shown Creative set the price of a click, but seven leads only show a direction.
Retail and ecommerce Proper Cloth
Click based analytics and Google view through reports disagreed on whether YouTube drove extra sales. Ran a five week geo holdout, with YouTube Demand Gen live in three states against a synthetic control. About $180,000 estimated sales lift (12.7%) Controlled When attribution systems disagree, design a counterfactual test and use the result to pick the next experiment, not a permanent verdict on the channel.
Media, social and ads Douyin (ByteDance)
Incentive offers had to be priced before the platform knew if users would finish ads and cover the payout. Used offline reinforcement learning with a world model to rank incentive policies before four online tests. Net profit per user up 7.96% Controlled Judge incentives on net profit at matched cost and treat ad exposure as a diagnostic.
Retail and ecommerce Zalando
Sales events lasted about six days but the pricing forecast ran weekly, and decisions took hours of manual work. Daily demand forecasts feeding an optimiser that balances revenue and long term profit. Profit contribution up 6% across 12 markets Controlled Choose a forecaster by the decisions it produces after optimisation, not its standalone error score.
Retail and ecommerce Taobao Live (Alibaba)
Compact AI hosts memorised skill names and prompt layouts, so every rule change risked breaking them. Trained a model on shifting skills, tools and prompts, with an editable rule set that operators update without retraining. GMV per participant up 4.33% across 1.98M users Controlled Move the sales playbook out of the model weights so operators change rules in hours, then test the whole system.
Retail and ecommerce Kuaishou
Designing recommendation experiments by hand was slow, and a win on one metric could quietly hurt the platform. An AI agent proposed new strategies, drawing on 310 past ones, over five rounds of live tests. GMV up 4.8% by the fifth strategy Controlled Give an experiment agent hard guardrails, because a win on the target metric can cost the rest of the platform.
Software and AI Wantedly
Candidates with few recent recruiter approaches churned more, yet ranking kept sending exposure to popular candidates. Boosted ranking for candidates with fewer than three recent scouts if they still cleared a match score threshold. Churn odds ratio 0.957, not statistically significant Controlled A directional result that misses significance still shows where the evidence chain breaks.
Other AMap (Alibaba)
The homepage already had a strong ranker for the next stop, which was hard to beat without slowing the app. Replaced the ranking stack with a small AI model that predicts the next stop directly. Active user rate up 9.67% over one month Controlled The most accurate offline model is not always the one to ship. The smallest version that fits the speed budget can be the right call.
Media, social and ads Kwai
Ads were marked as underperforming just for converting a little late. Retrained the scoring system to judge ads by whether a sale eventually happens, including late conversions. Realised ad revenue up 1.86% Controlled Split conversion prediction into two clocks so a late sale is not scored as a failed click.
Media, social and ads The Guardian
Most app users skipped sign in without a second thought. Made the skip option harder to spot on the first screen. Registrations and sign ins up 80%, agency reported Controlled Hiding the skip option lifted sign ins, but the case cannot say if those accounts stuck.
Media, social and ads Meta
Attribution credited ads for sales they had no real hand in, such as buying a different item. Changed how a sale is counted when someone clicks one item but buys another. Same item purchases up 1.27% in a two week test Controlled A conversion can be real while its attribution label is wrong. Separate same item and cross item purchases before optimising on them.
Media, social and ads ShareChat
Revenue experiments needed lots of traffic and time before results could be trusted. Grouped users by spending history before testing, to cut noise from heavy spenders. About 45% less traffic needed per revenue test Controlled When a few heavy spenders drive revenue variance, split users by spend history before testing instead of adding traffic.
Retail and ecommerce Bunzl
The most clicked homepage section was quietly steering shoppers the wrong way. Replaced that section with customer reviews. Sales up 9%, product page visits up 11% Controlled High clicks on a section do not prove it helps. Check what visitors do next before defending it.
Retail and ecommerce Sport Chek
Would a winning shipping message survive a higher spending requirement? Raised the free shipping threshold from $50 to $75 with the same message. Up 6.56% at $50, down 15% at $75 Controlled Retest the message when the offer changes. The previous winner no longer describes the same purchase decision.
Media, social and ads LinkedIn
Ads reached people likely to notice them, with little regard for who the ads could actually sway. Built a system that targets people the message has a real chance of changing. Long term value up 7.2% over eight weeks Controlled A response does not prove the message caused it. Target for incremental effect and withhold sends predicted to add nothing.
Software and AI Aurum (Astrea)
New trial users waited about two days before their data showed up. Cut the wait so real data appears the moment a trial starts. Trial engagement up 125% relative, 16 points Controlled Preloading data into the first session lifted activation, but engagement means little without downstream conversion.
Media, social and ads Pinterest
Countless photos had no clear reason to appear in search. Gave every photo an AI written description built for search. Visits up 18% in a four week test Controlled Writing the search intent for an image, not just its label, carried most of the extra traffic.
Software and AI Outrank
Could a paid leaderboard spot bring in sign ups that justify the cost? Spent about $12,000 on the placement and tracked trial sign ups. 53 extra trials on about $12,000 No control shown A before and after spike is a correlation, not a controlled test. The real result depends on how many trials become paying, retained customers.
Retail and ecommerce Cann
A top ad channel was suspected of looking better on paper than in practice. Turned the channel off across five states for 19 days to see what changed. About $90k estimated lift, range from a $152k loss to a $310k gain Controlled An inconclusive incrementality test still gives a better basis for a budget decision than a dashboard ROAS figure.

Swipe or scroll sideways to see the full table on mobile.

How to read this index

Each row is one Growth Signal story: the bottleneck, the change tested, the reported result and the limit on the evidence. Results are as reported by the company or agency involved and are not independently verified unless the story says so. Controlled means the source describes a control group or a randomised comparison. No control shown means the figure is a company report, a before and after change or an attribution count, which does not establish extra sales. Open a row for the test design, the source and the learning to carry into your next test.

Cite this data

Industry Contents, Growth Signal Index, IC Lab, https://industrycontents.com/ic-lab/growth-signal-index/, accessed [date].

Growth results can change as more evidence appears. Check the source story before quoting figures elsewhere.

Get the Growth hack digest

One growth experiment, its result and the learning to copy, in your inbox. The evidence limits are always stated.

How to read a growth experiment

What counts as a growth experiment?

A change to a product, page, message or channel, tested against a stated goal with a measurable outcome. A good one starts with a hypothesis and ends with a learning that decides the next test.

What is the difference between a reported gain and incremental growth?

A reported gain is a number a company published, such as attributed revenue or a higher click rate. Incremental growth is what would not have happened without the change, and only a control group or holdout can show it. The index marks which results had one.

Can I reuse these A/B test examples?

Reuse the idea, not the number. Your audience, traffic and offer differ, so treat each learning as a hypothesis to test on your own funnel.

Citing and reusing the index

Cite this index with credit and a link to Industry Contents. To copy, republish or redistribute the dataset, request written permission.

Read the Growth Signal stories · Explore IC Data Lab

Get TalkHead

Every Friday, TalkHead checks what your preferred AI tools cost, how businesses test their claims and what buyers should check before committing.

Unsubscribe anytime. Your email stays confidential.