A/B testing seems straightforward: take a landing page that already works, change one element, compare the results, and keep the stronger version.
In practice, even small changes can affect the entire funnel. Here are six real testing cases and the lessons they highlight.
The new version generated 35% less revenue. 26% fewer users started the survey, and 35% fewer reached the main exit.
The problem was the interaction: clicks that felt intuitive in the original version became less clear after the redesign. The test was stopped after two hours.
Lesson: after a redesign, make sure users still understand the core mechanic.
Instead, the new version generated 18% less revenue. Moving the age question to the first step almost doubled Age Exit traffic, but the number of users reaching the main exit fell by 30%.
A faster timer delivered +2% revenue, although the result varied across monetization zones.
Lesson: improving one stage of the funnel can hurt the final result. One test = one hypothesis.
One version generated 6% less revenue than the control, while the strongest version beat the original by just 2%.
The difference came down to a small UX detail: stones were highlighted slightly away from where users clicked. Around 80% of revenue came from the main exit, so even a minor drop in funnel completion affected the final result.
Lesson: small UX details can directly affect monetization.
Both simplified versions generated 10% less revenue, mainly because impressions in secondary monetization zones dropped by 11–12%.
Lesson: before removing an element, check how it contributes to the revenue flow.
After the data was segmented by traffic source, however, the difference almost disappeared. In CPA-based campaign models, the customized version actually performed better.
The team kept it for a separate traffic stream instead of abandoning it.
Lesson: an overall losing result doesn’t necessarily make a landing page useless. Check performance by source, GEO, campaign model, and traffic type.
Both versions quickly fell behind the classic landing page.What is more, 55–60% fewer users reached the end.
The new design also used smaller “next step” buttons, while the characters in the creative were less noticeable.
Lesson: CTA size, contrast, and visual hierarchy can matter as much as the main testing hypothesis.
Try ProPush.me now!
In practice, even small changes can affect the entire funnel. Here are six real testing cases and the lessons they highlight.
1. A new design that cut revenue by 35%
A working clicker landing page was adapted to a sports theme while keeping the same core mechanic.The new version generated 35% less revenue. 26% fewer users started the survey, and 35% fewer reached the main exit.
The problem was the interaction: clicks that felt intuitive in the original version became less clear after the redesign. The test was stopped after two hours.
Lesson: after a redesign, make sure users still understand the core mechanic.
2. One extra question — 18% less revenue
A multi-step survey was tested with additional questions added at the beginning. The goal was to increase interaction and improve segmentation.Instead, the new version generated 18% less revenue. Moving the age question to the first step almost doubled Age Exit traffic, but the number of users reaching the main exit fell by 30%.
A faster timer delivered +2% revenue, although the result varied across monetization zones.
Lesson: improving one stage of the funnel can hurt the final result. One test = one hypothesis.
3. Almost identical pages — 6% less revenue
Four versions of the same clicker landing page were tested, with the structure and mechanics kept almost identical.One version generated 6% less revenue than the control, while the strongest version beat the original by just 2%.
The difference came down to a small UX detail: stones were highlighted slightly away from where users clicked. Around 80% of revenue came from the main exit, so even a minor drop in funnel completion affected the final result.
Lesson: small UX details can directly affect monetization.
4. A “cleaner” page that lost 10% of revenue
Another test removed an element that appeared unnecessary. In practice, that element was moving users to an additional monetization zone.Both simplified versions generated 10% less revenue, mainly because impressions in secondary monetization zones dropped by 11–12%.
Lesson: before removing an element, check how it contributes to the revenue flow.
5. A losing landing page that still found its use case
A customized landing page was tested again after an unsuccessful first attempt. Overall, it still generated 33% fewer users reaching the main exit than the control.After the data was segmented by traffic source, however, the difference almost disappeared. In CPA-based campaign models, the customized version actually performed better.
The team kept it for a separate traffic stream instead of abandoning it.
Lesson: an overall losing result doesn’t necessarily make a landing page useless. Check performance by source, GEO, campaign model, and traffic type.
6. A Social landing with a weaker CTA
A Social landing with a pre-pop and segmentation question was tested against a simpler version without the pop-up.Both versions quickly fell behind the classic landing page.What is more, 55–60% fewer users reached the end.
The new design also used smaller “next step” buttons, while the characters in the creative were less noticeable.
Lesson: CTA size, contrast, and visual hierarchy can matter as much as the main testing hypothesis.
What these tests have in common
Several principles come up across all six cases:- Keep a live control.
- Start with a short trial to catch major drops early.
- Look at the whole funnel, not just one metric.
- Segment results by GEO, traffic source, device, and other relevant parameters.
- Check traffic anomalies that could affect the test.
- Change one variable at a time.
- Don’t draw conclusions from an insufficient sample.
- Don’t discard a landing page after one overall result — it may perform well for a specific segment.
- Don’t assume better UX means better monetization.
- Most importantly, understand why a variation won or lost.
Try ProPush.me now!
