Overview

In addition to subject line experiments, you can build multi-variant experiments within the body of a batch message using body experiments.

The ability to add a control variant at the message body level is only available in batch messages.

Body experiments are set up within the HTML content area of a message using Cordial's template syntax (Smarty).

Body experiments support the same conversion metrics as subject line experiments:

• Unique Opens
• Unique Clicks
• Revenue to Sent
• Order to Sent
• Revenue to Click
• Order to Click

Multiple experiments may be run concurrently in batch messages, but they must use the same testing strategy and populations (ex: strategy="split" or strategy="split-with-champion"). If you set your subject line experiment to "split with champion", your body experiment must also use the same strategy and populations.

Configure a body experiment

Split test without champion

The experiment element in batch messages

The variant element

• Test n variants where all variants are evenly split across the audience
• 50% A
• 50% B

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment}

• Test n variants where all variants are evenly split across the audience
• 33.3% A
• 33.3% B
• 33.3% C

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
   {variant name="C"}Test C{/variant}
{/experiment}

• Test n variants (same size) and 1 control as remainder
  • 5000 (10%) A
  • 5000 (10%) B
  • remainder (80%) C

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueclicks"}
   {variant name="A" sample-size="5000"}Test A{/variant}
   {variant name="B" sample-size="5000"}Test B{/variant}
   {variant name="C"}Test C{/variant}
{/experiment}

• Test 1 single variant (same size) and 1 control as remainder
• 5000 (10%) A
• remainder (90%) B

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueclicks"}
   {variant name="A" sample-size="5000"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment}

Split test with champion

Not currently supported with messages scheduled to time zone of the recipient.

• Test n variants and auto-send champion after x mins/hours
• 2500 sent A
• 2500 sent B
• Remainder sent to the champion after three hours.

{experiment name="Split-with-Champion" conversion="uniqueclicks" strategy="split-with-champion" send-champion="max-time" max-time="3h" sample-size="5000"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment} 

• Test n variants where all variants are split by a specific percentage and auto-send champion after x mins/hours
  • 10% A
  • 10% B
  • Remainder sent to the champion after two hours. 

{experiment name="Test" conversion="uniqueclicks" strategy="split-with-champion" send-champion="max-time" max-time="2h0m" sample-size-type="percentage" sample-size="20%"}
   {variant name="A"}A{/variant}
   {variant name="B"}B{/variant}
{/experiment}

Body experiment FAQs

Conditional logic with experiments

Experiments and Smarty run in parallel and asynchronously. Therefore it is not possible to use conditional logic to affect content in a message based on experiments.

For example, the following conditional code will not work as intended because the variable for $gender will always equal boy in this case:

{* will not work *}
{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
{variant name="A"}{$gender = "girl"}Hello girl,<br><br>{/variant}
   {variant name="B"}{$gender = "boy"}Hello boy,<br><br>{/variant}
{/experiment}

{if $gender == "girl"}Girl content{elseif $gender == "boy"}Boy content{/if}

The recommended way to write this would be to put all desired content within each variant:

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Hello girl,<br><br>Girl content{/variant}
   {variant name="B"}Hello boy,<br><br>Boy content{/variant}
{/experiment}

You also have the option to place each content piece within an HTML content include. This improves usability when working with large amounts of code within the HTML editor.

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}{include 'content:GirlContent'}{/variant}
   {variant name="B"}{include 'content:BoyContent'}{/variant}
{/experiment}

In the next article, you can learn about batch experiment performance.