Follow Along CPV Lab Pro - Multivariate testing with a single landing page

[CPV Lab Pro]

Hi friends,

We have been asked many times about the easiest way to perform multivariate testing of landing pages. Something that will work when testing small page variations just to get information about how each small change performs.

And of course, you can create separate landing pages with slight variations, then split-test traffic between them in order to see which ones perform better.

But there is an easier way to do multivariate testing with a single physical landing page, without generating more pages. All you need is a multivariate testing tool which can rotate variations directly in the page, then report results back to the tracker.

Let’s say that you want to test 3 headlines, 2 images and 3 buttons (CTA). This means that you will have to generate 3x2x3 = 18 different pages for such variations.

But what if there was a way to test all these 18 page variations within a single landing page? You would probably think that it must be complicated or that it must require coding skills… Well, this is super-easy and it can be done with no coding at all in MV Lab.

I will be doing a follow-along in this thread with a real campaign in order to show how MV Lab can be used and what the multivariate testing results are.

What I’m looking for specifically is how to track multivariate results and the CTR of different page versions.
I’m absolutely sure that the campaign won’t be profitable and that the offer chosen is not one of the best, but this campaign in not about making $$$, it’s just for learning multivariate testing in an easy way.

So my focus is on the landing page and how to adjust it based on the metrics tracked.

Traffic Source: PropellerAds
GEOs: Canada and Australia - I've opted for this to have the landing in English
Tracking Tool: CPV Lab Pro + MV Lab
Affiliate Network: Traffic Company
Type of Offer: not great, not terrible
I believe this campaign will be successful because: Well, I don’t expect this campaign to be successful because it is only designed to test multivariate features in the landing page and not for big success.
I am unsure about: the Profit and the ROI?

My initial landing page is pretty simple and it looks like this:

I want to see how small changes in this page will improve CTR and I plan on perform multivariate tests for the following items:
1. Subtitle text – test 3 variations: “BLACK FRIDAY”, “NEXT 10 MINUTES”, “YOUR LOCATION”
2. Background image – test 2 variations: with background image and without background image
3. Button color – test 3 colors: “LawnGreen”, “Yellow, “Red”.

These 3 will be the snippets from my page that I will want to rotate multiple variations for.

Without MV Lab, the difficult approach would involve generating multiple pages, one for each combination, but as I mentioned before this will lead to 3x3x2 = 18 pages.

So, let’s see how the MV Lab approach is and how much time this will save us.

There are a few requirements for using MV Lab for multivariate testing:
- You need to have access to the page HTML code, so you need a page that is hosted on your server and not a page from a landing page builder tool (but a little birdie told me that a version compatible with Wordpress is being prepared )
- You need a PHP page – if you have a page with *.htm or *html extension, then you can simply rename it to have a *.php extension
- The mvlab.php file generated in your MV Lab account should be uploaded on the server, in the same location as your landing page
- CPV Lab Pro as the tracker in order to see the results of your tests directly in the tracker reports.

I have created my campaign in CPV Lab Pro just as any regular landing page + offer campaign. I won’t describe the entire campaign setup process here, but if you are interested in documenting this, let me know and I will open a new thread.
There was one extra thing to do in my campaign: I have setup one extra token for each of my snippets in the multivariate test. These extra tokens will receive information about the variations displayed for each visitor.

Now using the visual editor for MV Lab included in CPV Lab Pro, I have defined each snippet and the variations that I want to be rotated for it. I can also set a Share (number of visitors) for each variation, but for now I left the Shares blank in order to have traffic distributed evenly.

There is a "Generate Code" at the end of this page that will provide the exact code needed in my landing page.

I’ve copied the code and pasted it at the beginning of the HTML code in my landing page (before the <html> tag and any other tags). This code will quickly execute before the page is loaded and set the variations for each visitor.

While editing the landing page to add the code, I have also defined the places where I want the variations to be rotated.
All I had to do is replace the actual fields with the snippets.

These will be the 3 snippets and I need to use the wildcard ##snippet_X## (where X will be the snippet ID) in the exact place where I want it to be replaced by the actual variation.

I placed ##snippet_1## where I want the subtitle text to appear:

I placed ##snippet_2## in the <style> tag that controls the background image (2 places):

And I placed ##snippet_3## as the background color and border color for my button:

Now the configuration of MV Lab in my landing page is complete. All I have to do is to upload the landing page on the server and it’s ready for traffic.

You can see my page at the URL below. Refresh the page several times to see how different variations will appear in the 3 snippets with each page refresh… this is how the page variations will be rotated between my visitors (and yes, the url below has a token to track how many times you clicked it )

https://win-money-online.net/lucky-tiger-promo.php?src=afflift

I have already started traffic in this campaign and I will post here the initial results of these multivariate tests.

We will see which variations perform well and which don’t, then adjust the landing page according to those details. The goal is to make adjustments that will increase the CTR on the page.
Oh, and a few conversions would also be fine, but the focus here is on the page CTR for each variation.

Let’s see how this one goes! I will share the initial results soon.

Cheers,
Radu

Here is a preview of some variations once the page is refreshed

 

[Luke]

This is super interesting, Radu. I am excited that you are sharing it with us.

What type of traffic are you using from PropellerAds?

 

[servandosilva]

Nice! Very convenient for strict traffic sources that don't allow redirects or multiple landers.

 

[CPV Lab Pro]

What type of traffic are you using from PropellerAds?

Started this test with Push traffic from PropellerAds.

Nice! Very convenient for strict traffic sources that don't allow redirects or multiple landers.

Indeed @servandosilva, this method works very well in non-redirect campaigns where you link directly to the landing page.

 

[Luke]

Started this test with Push traffic from PropellerAds.

Nice. Good luck! Looking forward to some results

 

[BennyBall]

Started this test with Push traffic from PropellerAds.



Indeed @servandosilva, this method works very well in non-redirect campaigns where you link directly to the landing page.

Does CPVLab have redirectless tracking? Like you can send traffic like this:

"httpx://mylanding.com/lander1.html?source=xx&zone=xx"

and it will grab the query parameters and set a subid for that visitor?

 

[CPV Lab Pro]

Does CPVLab have redirectless tracking? Like you can send traffic like this:

"httpx://mylanding.com/lander1.html?source=xx&zone=xx"

and it will grab the query parameters and set a subid for that visitor?

Yes @BenitoB, it does have redirectless tracking!

You can perform tracking directly in the page and capture all parameters passed in the landing page URL. This method uses the Direct Traffic code placed in the landing page.

You will have all the metrics available in reports just like when tracking with the Campaign URL.

 

[CPV Lab Pro]

Ok, after running a few hours of traffic in this campaign, I have the first results.

As mentioned before, I will be focusing on the Clicks from the landing page and the CTR of the page for different variations of the 3 snippets that were defined. I’m not focused yet on conversions, I just want to increase the page CTR.

Let’s see the reports from CPV Lab Pro for my multivariate testing in this campaign. The initial overall results of the page are as follows:

So, that’s a 8.33% CTR for my page across all variations... which is not too great.

But let’s dig deeper and see how my multivariate test went.
Let’s look first at the first snippet which rotated 3 texts for the subtitle: “BLACK FRIDAY”, “NEXT 10 MINUTES”, “YOUR LOCATION”.

Well, this brings some information into what happened in my landing page. I can see that 2 of my Subtitle variations were getting higher than average CTR, while the third variation wasn’t performing at all.

Looks like Subtitle 3 is a certain candidate for removal from my landing page.

Let’s look now at the second snippet, the background of the page, which I’ve tested in 2 variations: with and without a background image.

Just like with snippet 1, we have a clear underperforming variation here, the one without a background image (Image 2 above). Not as drastically, but still pretty big difference between the 2 variations from here.

Let’s combine snippet 1 and snippet 2 in our report now and see how they performed together.

This report offers more details about the correlation of my first 2 snippets. The Subtitle 1 variation was doing well no matter what Image variation it was paired with and also Image 1 was doing fine with the first 2 variations for snippet 1 (Subtitle 1 and Subtitle 2).

The clear loser from this report is Subtitle 3 which didn’t perform with any of the 2 Image variations.

Let’s check now the third snippet from my test which is the button background color. This was tested in 3 variations: “LawnGreen”, “Yellow, “Red”.

As we can see in the above report, the CTR for each of the 3 variations for the button background is pretty similar, there is a very small variation here. This means that the button background color didn’t really have an influence of the clicks received in my page, so I should focus on the other 2 variables (Subtitle and Image).

If I run a combined report for the 3 variables (Subtitle, Image and Button) sorted by CTR descending, then things look like this:

First Conclusions:

Based on this initial traffic, I’ve identified that the third variation for the Subtitle snippet (“YOUR LOCATION”) isn’t working at all, no matter what other variations it was paired with. This is a good moment to remove this variation from the landing page as it isn’t bringing results.

Also there is a clear difference between the background image variations. The variation containing a background image performs better and traffic should be increased to it. Or maybe the variation with no background image should be removed.

The 3 background colors for buttons didn’t make a difference for Clicks and CTR, so they can be kept as they are.

Next Actions:

Normally I would remove the third variation for the Subtitle snippet, but instead I will change it to something else in order to test it further.

I will change this variation from “YOUR LOCATION” to “<visitor’s city>” where this value will be dynamic based on the actual visitor location. Let’s see if the CTR will improve when I show the exact visitors’ location in the landing page instead of the generic “YOUR LOCATION” text.

It’s pretty easy to write the actual location for visitors in my landing page with MV Lab.

First I will configure CPV Lab Pro to pass visitors’ city in the landing page URL in the “city” parameter by ending my Landing Page URL with “?city={!mm_city!}”. The {!mm_city!} placeholder will get replaced by the actual city of each visitor.

Then I will change the variation for MV Lab to read from that “city” parameter as this:

Notice that I’ve also set “YOUR CITY” as the default value in case the city cannot be determined for a visitor.

Now MV Lab will write the actual city for each visitor when this Variation 3 will be displayed.

Another change I will make will be regarding the background image snippet.
I’m not removing yet the variation with no background which performed worse, but I will be sending more visitors to the other variation (with background image). I’ve set 75% of traffic to go to variation 1 (with background image) and 25% traffic to go to variation 2 (no background):

Snippet 3 (background color for button) will remain unchanged, and I will continue to monitor it.

I’ve implemented these changes using the visual editor for MV Lab from CPV Lab Pro and then added the newly generated code in my landing page. Then I’ve uploaded the landing page on my server and all new visitors will have the page displayed with the new settings.

Hopefully my CTR improves now after these changes.

I will analyze the results again tomorrow and post an update here to see what changed.

Cheers,
Radu

 

[DanaP]

Will this MVLab & lander work with Google campaigns?

 

[CPV Lab Pro]

Will this MVLab & lander work with Google campaigns?

Sure, such landing pages that use MV Lab will work on traffic sources that require Parallel Tracking and no-redirects between the ad and the final URL.

So it will work well for Google Ads and Bing Ads campaigns.

 

[topspot]

I've been with CPV Lab since the "old days" where things got a bit messy at one point, but now CPV Lab Pro is just a beast of a tracker which just keeps on getting better, for instance their latest update to v8.1 now includes the ability to see all the details of a single click https://doc.cpvlab.pro/click-lookup.html love it Radu!

 

[CPV Lab Pro]

Hi there!

I've returned with a new round of results from our multivariate test with MV Lab from CPV Lab Pro.

Yesterday I’ve made a few changes in the variations as an attempt to increase my page CTR, then continued to run traffic to my page.

I switched from a generic “Offer available in your location” text to a dynamic text that included the actual city of visitors: “Offer available in <actual-city-here>”. Would this make a difference?

Let’s see first the overall results for today’s traffic:

First thing that I’ve noticed here was that the overall CTR really increased from 8.33% to 15.63%! That’s a nice improvement, almost double, but I want to dig deeper into the data to see where it comes from.

So I’m checking the first snippet from my page which is the Subtitle text. This rotated 3 variations for the subtitle: “BLACK FRIDAY”, “NEXT 10 MINUTES”, “<actual-city-of-visitor>”.

And I’m really impressed with what I see here. Variation 3 for Subtitle got up to 20% CTR? So the change made yesterday to display the actual city of visitors in the page really worked!

That variation 3 for Subtitle was something that I initially considered removing all together, but then decided to just modify it and this performed well. It was the top performing subtitle today and had a 13 times increase in CTR from yesterday (up to 20% from 1.54%). That’s really good!

The other 2 subtitles have also increased in CTR, but just marginally compared to yesterday.

Let’s see now the second snippet, the background of the page, which I’ve tested in 2 variations: with and without a background image.

Yesterday I’ve changed from 50%-50% traffic to these 2 variations to 75% of traffic for the background image variation and 25% of traffic for the no-background image variation.

I see that both variations show an increase in CTR, although the second variation (no background) image continues to perform worse, just like it did from the beginning.

Now let’s add both Subtitle and Background image in our report and see their combinations performing:

As expected based on the individual reports, the best performing option is the actual city as Subtitle and the background image variation. Combined, these 2 got up to 22% CTR. Closed second comes the Black Friday Subtitle and the background image with 19% CTR.

Looking at the bottom rows in this report it’s clear once again that the variation without a background image is not performing and also the Next 10 minutes Subtitle has low click figures.

Now moving to the third variable in my test, the background color of the CTA button. Results look like this:

The CTR figures are very close again for this variable, as it was yesterday. These variations don’t make much difference in the context of my landing page and it’s not possible to identify a winner here.

The combined report for all 3 variables in my test looks like this:

Conclusions & Actions:

The change to display the actual city name for each visitor proved successful as this made the corresponding variation the top performer for the Subtitle snippet. This change will remain in my campaign.

But the “Next 10 minutes” variation for Subtitle remained with a pretty low CTR, so it looks like a good moment to drop it from my page.

For the second variable that I’ve tested, the background image, it was confirmed once again that the variation with no page background has poor results. I will drop this variation from my page and keep only the option with a background image.

As for the third variable, the background color of the CTA button, all 3 variations performed pretty similar and this makes me keep all 3 variations in the landing page.

My overall CTR increase from the page can be attributed to the better performing variation in Subtitle and also to the fact that I’ve set the background image variation to be displayed to more visitors than initially (75% of visitors). Both these changes proved successfully and had a positive impact on the CTR.

In CPV Lab Pro, the new MV Lab configuration for the page based on today’s findings will be as follows:

Keeping only the variations that performed well should allow me to get an even better CTR in my page in the next days.

The current improvement compared to those initial results is significant and I’m really pleased to see that such small changes in the page led to better results.

I will keep you posted with the next results from the test.

Cheers,
Radu

 

[Jimmy]

Brilliant FA @CPV Lab Pro

I believe this campaign will be successful because: Well, I don’t expect this campaign to be successful because it is only designed to test multivariate features in the landing page and not for big success.

lol sounds like the last couple of FAs I've done

 

[CPV Lab Pro]

Hi all,

Now that all the craziness with Black Friday has passed, I want to follow-up on our multivariate test with one more thing that I've noticed while making the previous changes.

I have also tracked the Engage Rate for my landing page in this campaign.

I have configured the Engage event to be fired when a visitor stays for at least 5 seconds on the page.
So the Engage Rate (Engage% in reports) will be the percentage of visitors that spent at least 5 seconds on the landing page.

Initially my landing page got an average Engage Rate of 35.94% across all variations, with better results for the variations with higher CTR.

This was pretty low and I felt that it could be improved.

In the combined report for all 3 variations, I sorted them by CTR to see what are my TOP Variations (the ones with CTR above average). And I got this:

The changes described in previous posts which were made to increase the CTR of the page also helped the Engage Rate of my page.
The Engage Rate jumped up to almost 50% after applying the changes, with top CTR variations also having top Engage Rates:

What is worth mentioning here is that the Engage Rate (visitors who spend a specific time on the page) is also an important metric as this can offer additional information when performing multivariate testing of pages.

So, track the Engage Rate of your pages every time you can for extra information!

Cheers,
Radu

 

[Arif]

That engage metric is one I still miss still from when I last used CPVLab.

Just curious, you made any leads from this follow along?

 

[Epocks]

I currently capture email on my landing page via having my Aweber account integrated through my landing pages builder inside of wordpress, will this still work if I create variations of the landing but also collect emails?

 

[CPV Lab Pro]

Multivariate testing with MV Lab will work fine with an opt-in form in the page (Aweber or anything else), everything will be configured in the same way.

What you need it to actually have access to the page HTML in order to place the MV Lab code in it for the multivariate tests. If using a landing page builder tool, then most likely you won't be able to place the custom MV Lab code in the page source. So I recommend that you try to download the HTML of your landing page and use it as a standalone page in order to take advantage of the MV Lab features.

 

[Epocks]

Multivariate testing with MV Lab will work fine with an opt-in form in the page (Aweber or anything else), everything will be configured in the same way.

What you need it to actually have access to the page HTML in order to place the MV Lab code in it for the multivariate tests. If using a landing page builder tool, then most likely you won't be able to place the custom MV Lab code in the page source. So I recommend that you try to download the HTML of your landing page and use it as a standalone page in order to take advantage of the MV Lab features.

I am using a landing page builder inside Wordpress, so I am able to download the source code and add the MV Lab code quite easily but how do I then tern that HTML source code into a PHP code?

 

[CPV Lab Pro]

If you can download the page source, then you can simply rename the page to *.php instead of *.html.

You can place all this HTML code inside the *.php page, then add the MV Lab specific code (generated with the visual editor from CPV Lab) at the top of this page.

That's all, you will have multivariate testing configured in your page.

 

[Epocks]

If you can download the page source, then you can simply rename the page to *.php instead of *.html.

You can place all this HTML code inside the *.php page, then add the MV Lab specific code (generated with the visual editor from CPV Lab) at the top of this page.

That's all, you will have multivariate testing configured in your page.

Ok, I didn't realise it was that simple. I will give it a crack and let you know how it goes, thank you