Creating several variations of a lightbox with the same conversion goal is an excellent way to determine which type of lightbox message leads to the best overall performance. Cordial's Primer Webforms Lightbox Variations combined with the A/B Testing Wizard allows you to easily set up a testing environment, quickly assess performance, and immediately exploit a winning formula. This article serves as a tutorial for the A/B Testing and Lightbox variation features.
In order to run an A/B test, you first create a variation of your lightbox. To do this, log into to your Primer account and click +Add in the variations column.
After clicking +Add, a pop up modal presents three options for creating lightbox variations.
The options are:
- New Variation From Original
- New Variation From Theme
- New Variation From Another Lightbox
Option 1: New Variation From Original
This option creates a carbon copy of your original lightbox and adds it as a variation. After selecting New Variation From Original you will be taken back to the lightbox management interface. The Variations column will now have an Edit link. Use this link to view and customize the lightbox variation.
Option 2: New Variation From Theme
This option creates a new variation from one of the pre-existing lightbox themes available in the gallery.
Option 3: New Variation From Another Lightbox
This option allows you to create a variation from an existing lightbox.
Setting a lightbox as the Primary
When a lightbox with variations is utilized outside of an A/B test, only one version of the lighbox is shown. The lightbox that is displayed is the Primary lightbox. By default, the original lightbox is considered the primary. To make a lightbox variation serve as the primary, click OFF in the Primary column. You will see OFF change to ON.
You must publish the lighbox in order for the Primary change to take affect.
Once you have created variations of a lightbox, you can perform a test between any two variations. The test can be between the original lightbox and a variation or between a variation and another variation. To conduct a test, log into Primer Webforms and select A/B Tests from the A/B Testing drop down menu.
You will then be taken to a page that allows you to see the performance of your existing A/B Tests. To start a new test click on +New A/B Test
Inside the Wizard you can start a new A/B Test. Follow the steps provided by the Wizard to perform the following:
- Name the A/B Test
- Select the lightbox you would like to test – Note: Only lightboxes with variations will show up in the selectable list
- Pick the lightbox variations – Remember you can only test two at at time
- Select an impressions limit – This defaults to 500, but it is recommend that the value be increased to at least 1500. It's a good idea to look at historic page view data to get a sense of what the impression limit should be.
- Select a date range – A date rage of at least 7 days is recommended.
Once you've completed the wizard you must publish changes in order for the A/B Test to start running. A/B Tests that are live will show up with a checkmark next to them in the Live column in the A/B test section.
- You cannot delete lightboxes that have active A/B tests
- You cannot create more than one A/B test for the same lightbox
- You can only test design formats with A/B tests. To test Conditions (e.g., three second timer vs. Exit Intent) you must use the multi variate testing features.