Multivariate Testing (MVT) makes it possible for you to test multiple sections on a page/screen to determine which combination of sections are most successful in achieving a desired outcome. MVT is similar to A/B testing in that website visitors are unknowingly shown different web page variations, known as Experiences, but A/B tests only measure the impact of a single section change. Since multiple sections of the page are tied to a specific goal, like driving conversions, you only need to run one MVT instead of several A/B tests.
This article will explain how MVT works and review the process for using it in Evergage.
How does MVT work?
Suppose you are uncertain which header (header 1 or header 2) and footer (footer 1 or footer 2) on your company's B2B website is the most effective at driving leads. Using MVT, you can simultaneously test these options to determine which combination works best.
What is the downside?
Because an MVT campaign will split traffic up across all of the potential combinations, it may take some time to reach statistical significance depending on the amount of traffic you have on your site. For example, with a 50/50 A/B test and a 10% control, 45% of your users will see Test A, 45% will see Test B, and 10% will see nothing. But a four combination multivariate test with a 10% control will split traffic between 22.5% of your audience for each combination. Referencing the image at the right, 22.5% of visitors would see each of the four Test Combinations. For more information, please refer to the article on Campaign Statistics -- Confidence.
Additionally, it is more difficult to see when one section impacts conversion, but not other parts of the multivariate test.
Use Multivariate Testing in Evergage
Provided your website receives enough traffic to successfully validate each test combination, you will know with confidence which header and footer combination drives the most leads.
Use different tabs to organize each element to be tested. You can change up to three sections per page and test up to nine combinations of a page at one time.
- Create or Edit a Campaign
- Select EXPERIENCES
- Select Multivariate as the Test Mode, which will change EXPERIENCES to SECTIONS
- Name the first section or element to be tested. In this example, the first section is Headers
- Click Add Section to add additional sections. In this example, the next section is Footers
In this example, there are two variations for each section: two different headers and two different footers. In this set of steps, you will create those variations.
- Still in > SECTIONS, select Variation 1 for the first section (Headers in this example)
- Change the Name. In this example, Header 1 is the new variation name
- Click New Message which will create a Popup message as the default
- Click Message Types for more information to change the message type, if needed, in Message Settings. This example is an Infobar. See
- Click OK. SETUP will close and you can style the message as desired
- Add and configure additional variations in
You may need to delete a variation to adjust your testing model. As with other MVT configurations, this must be done in SETUP.
- Select SECTIONS
- Click next to the variation you wish to delete
Adjust User Percentages
After your sections and variations are configured, you need to adjust the percentage of users that will see each variation as well as the percentage that will see the control, if you are using one.
- Select SECTIONS
- Use the slider to adjust the percentages for each variation. Since percentages have to add to 100, other variations will adjust as you move the sliders
Adjust the default Control % as needed
Another ApproachIn this example, if you wanted to test a header without a footer or vice versa, you could add a header and/or footer variation without a message and set the Control % to 0
Restrict to Segments of Users
As with any Evergage campaign, you can restrict MVT campaigns to one or more segments of users with a Who Sees This campaign-wide rule. Refer to Add Rules for Campaigns, Experiences, and Messages for information on creating this type of rule and Create a Segment for information on creating segments of users.
Test and Publish Your Campaign
After you have completed your MVT campaign, refer to the Test Campaigns article for instructions on testing and publishing.
Determine the Best Approach
Evergage Campaign Statistics will help you identify the best combination of elements to achieve your goal. Refer to the Campaign Statistics Overview article and related articles for more information.
This page has no comments.