10 min read
Algolia used AI to build a pricing page test in hours. Amplitude reports a 15% lift without naming the target metric.
TL;DR
- Algolia’s website team wanted to test its pricing page, but the change needed custom JavaScript, CSS and HTML while its small development team worked on higher priorities.
- Website optimisation manager Brittany Cole described the change to the Amplitude Website Conversion Agent, which produced several previews and set up two live variants within hours.
- Amplitude says the test lifted Algolia’s target metric by 15% and estimates a six-figure pipeline effect. It does not name the metric or publish the variants, baseline, sample, duration, allocation or statistical uncertainty.
- The copyable lever is to let AI build the variants while people keep the hypothesis, QA, launch and decision.
Brittany Cole, website optimisation manager at Paris-founded AI search company Algolia, had a pricing page idea and a familiar problem. The test needed custom JavaScript, CSS and HTML. The developers who could write it had more important work, so a change that looked small from the website team’s side could wait weeks or months before a visitor saw it.
A small website idea can have a long wait for a developer. Algolia wanted to shorten that wait while keeping a person in charge of what went live.
Amplitude’s account of the experiment, published on 26 June 2026, says Cole used the Amplitude Website Conversion Agent to do the building. She described what she wanted, reviewed working previews and iterated, and the agent created two live variants ready to launch within hours. Cole says the normal route would have meant “weeks, maybe months”. Amplitude, the software company behind the agent, reports that the AI pricing page experiment lifted Algolia’s target metric by 15% and carried an estimated six-figure pipeline impact. The account comes from Amplitude’s first AI Impact Awards, where Algolia placed second out of 31 submissions, and it does not name the metric, so the operating change is easier to inspect than the revenue claim.
Please join the queue
Algolia was founded in Paris and later moved its headquarters to the United States while keeping a Paris presence. Its website has to explain a metered technical product to developers and enterprise buyers.
The current Algolia pricing page asks visitors to think about search requests, records, product capabilities and enterprise plans. That says nothing about the exact version tested, since Amplitude never publishes the control or the treatment. It does show why changes on this surface can outgrow a visual editor. A pricing experience combines interaction logic, brand rules and a commercial event that may only happen later in a sales pipeline.
The usual route ran through a development ticket, a review cycle and a test configuration. Each step had a reason to exist. Together, they made a small experiment a substantial request.
Amplitude does not state Algolia’s formal hypothesis, so this one is a reconstruction. If the website team could use an AI agent to build brand-compliant variants and keep human approval before launch, it could run a pricing page test without waiting months for engineering capacity.
Built, pending approval
The agent did more than suggest copy. In Amplitude’s account it produced several working variants, took Cole’s feedback on how they looked and behaved, and created the experiment with two live variants. Amplitude says the generated code matched Algolia’s brand guidelines and worked as intended, and Cole shared preview links with stakeholders hours after she had the idea.
Amplitude’s product documentation shows the wider workflow. A user picks a conversion event and a page, the agent analyses captured page data, proposes strategies and builds variants for the chosen one, and the operator can preview, remove or regenerate a variant with feedback. The agent then saves a draft experiment, and a person opens the experiment view and starts it. The agent gets the experiment ready. The person still starts it.
The setup still needs instrumentation. Amplitude says the page must carry its experiment SDK, a small code package that lets the page talk to Amplitude, or its Chrome extension for previews, and Experiments must be switched on to create a test. Its web experiment documentation also recommends previewing every variant before exposure.
15% of what?
The reported 15% lift comes with an estimated six-figure pipeline effect. Amplitude does not name the target metric, which leaves quite a lot hiding behind a small percentage sign.
For anyone tracking pricing page conversion, the phrase target metric carries nearly all the ambiguity. It could mean a pricing page click, a form completion, a qualified opportunity or another event, and the source does not say. The baseline, absolute counts, audience, market, sample size, run length, traffic split, stopping rule and confidence interval are also missing. A relative lift is the common reading of a figure like this, though the source does not confirm it.
Amplitude describes neither the control, the difference between the two variants nor which one won. That leaves a reader unable to copy the customer-facing change or to check whether the pipeline estimate came from observed opportunities, an attribution model or a projection from the target event. Independent replication has not turned up in the public record. Amplitude supplied the software, ran the awards programme and published the result. Algolia is named, Cole is quoted and her LinkedIn profile confirms her role at the company.
A draft beats a ticket
Before the test had a result, the website team had something it could put in front of colleagues. Algolia changed who could do the scarce work. Its developers no longer had to be the only people who could turn a website hypothesis into executable code. The website operator could produce a working draft, gather stakeholder feedback through previews and bring engineering in for the parts that justified it. Growth Signal covered a similar move at Robinson Club, where AI built landing pages in seconds instead of days.
An agent can make five weak ideas runnable faster than a team once made one. Decide what would count as a useful result before asking it to build. Write the problem, hypothesis, primary metric, guardrails and decision rule before asking the agent to build a variant.
The split also gives technical teams a cleaner review target. Instead of turning a loose marketing request into production code, they can inspect a concrete draft for selectors (the page elements a test targets), responsive behaviour, accessibility, analytics events, content security rules (the browser limits on which scripts may run) and rollback behaviour.
European operators should add consent and regional data controls to the same design. Amplitude’s implementation documentation offers an EU data centre script and says sites that require cookie consent should gate storage before loading the experiment script. Algolia’s case says nothing about its market or privacy setup, so treat those controls as implementation requirements and leave them out of the reported experiment.
Try one blocked idea
You do not need an ML team to copy the lever. Start with one experiment that has approval in principle and is blocked mainly by frontend capacity. Avoid checkout logic, authentication and other surfaces where a generated change can create financial or security risk. A pricing, upgrade or lead generation page with stable instrumentation is a better first boundary.
Before generation, write a one page test contract that names the page, the customer problem, the exact intervention, the primary metric and two guardrails. Add the minimum detectable effect, the smallest lift the test is built to notice, along with the planned sample or duration and the person who can stop the test. If those fields are unknown, the agent is being asked to automate ambiguity.
Then give the AI the design system, approved components and analytics taxonomy, and ask for a draft and previews rather than a launch. Review the output on desktop and mobile, confirm that the exposure event fires once, check that the conversion event points at the business outcome you intend, and make sure the control stays untouched.
After the test, record the commercial effect and the production cost. The primary outcome might be qualified pricing enquiries or pipeline created per exposed account, and the operating outcome might be hours from approved hypothesis to live experiment plus engineering hours consumed. If the variant wins but review effort grows enough to erase the saved time, the workflow has only moved the bottleneck.
Test the builder too
Amplitude’s public case says nothing about what happened next. The strongest follow-up would compare the production method itself. Randomly assign a backlog of similar web experiments to the normal development route or the AI-assisted route, then compare time to launch, engineer hours, QA defects and the share of tests that reach a decision. That would show whether the shorter queue produces more completed experiments, or simply gives reviewers more drafts to check.
The implementation question also runs through Optise’s managed website offer: who checks a proposed change before it reaches customers? The two accounts cover different products, but both distinguish producing a change from taking responsibility for it.
| Experiment detail | What the reporting shows |
|---|---|
| Change | Website optimisation manager Brittany Cole used Amplitude’s Website Conversion Agent to generate, iterate and configure two live variants, while retaining human review and launch control. |
| Reported result | Amplitude reports a 15% lift in Algolia’s unnamed target metric and an estimated six-figure pipeline impact. |
| Evidence | Amplitude-published award case quoting Algolia website optimisation manager Brittany Cole, supported by Amplitude product documentation and Algolia company pages. Not independently replicated. |
| Limits | Published by Amplitude in an award account. The target metric, variants, control, baseline, sample, duration, allocation, market and statistical uncertainty are not disclosed. |
Sources and disclosure
- Amplitude AI Impact Awards account of Algolia’s experiment, published 26 June 2026
- Amplitude Website Conversion Agent documentation
- Amplitude Web Experiment setup and preview documentation
- Algolia company profile and current pricing page
- Brittany Cole of Algolia on LinkedIn
Amplitude published the case and supplies the software, so the figures come from the company. The 15% lift and the pipeline estimate come from its award account and have not been independently replicated in the public record.
Related reading
- How Fyxer used AI to run 360 growth tests with just four engineers
- Aurum’s free trial started with a 48-hour wait
- AI is repricing Nordic SaaS. Who keeps the value?
FAQ
Which page did Algolia test?
Amplitude says Algolia ran an experiment on its pricing page that needed custom JavaScript, CSS and HTML. It does not disclose the control, the customer-facing changes or the difference between the two generated variants.
Did the AI launch the experiment by itself?
No. Amplitude’s documentation says the agent creates a draft experiment after a person reviews the variants, and a user still opens the experiment view and starts the test.
How strong is the 15% result?
It is a result published by Amplitude and tied to a named company and practitioner. The source does not identify the target metric or publish the baseline, sample, duration, allocation or uncertainty, so the figure cannot be independently evaluated.
Can another growth team copy this?
Use AI to turn an approved hypothesis into reviewable variants while people keep metric selection, QA, launch and decision rights. Measure engineering hours and defects beside the commercial outcome.
Compare more AI experiments and their evidence in the Growth Signal Index, browse the Growth Signal archive, or use the benchmark tools in the Industry Contents Lab.