How can we help?

Body Experiments

Overview

In addition to subject line experiments, you can build multi-variant experiments within the body of the message using body experiments.

Body experiments are set up within the HTML content area of a message using Cordial's template syntax (Smarty).

Body experiments support the same conversion metrics as subject line experiments:

  • Unique Opens
  • Unique Clicks
  • Revenue to Sent
  • Order to Sent
  • Revenue to Click
  • Order to Click
Note: Multiple experiments may be run concurrently in batch messages, but they must use the same testing strategy and populations (ex: strategy="split" or strategy="split-with-champion"). If you set your subject line experiment to "split with champion", your body experiment must also use the same strategy and populations.

Configure a body experiment

Split test without champion

The experiment element in batch messages

Attribute Expected Required
name string true
strategy split-with-champion
split
true
conversion uniqueclicks
uniqueopens
totalrevenue/totalsent
totalorders/totalsent
totalorders/uniqueclicks
totalrevenue/uniqueclicks
true
sample-size integer If strategy=split-with-champion
sample-size-type string If strategy=split-with-champion
send-champion max-time If strategy=split-with-champion
max-time 30m
4h
If strategy=split-with-champion

The variant element

Attribute Expected Required
name string true
sample-size integer False - if defined, all variants for the experiment must have the same value, barring the control.
  • Test n variants where all variants are evenly split across the audience.
    • 50% A
    • 50% B
{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment}
  • Test n variants where all variants are evenly split across the audience.
    • 33.3% A
    • 33.3% B
    • 33.3% C
{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
   {variant name="C"}Test C{/variant}
{/experiment}
  • Test n variants (same size) and 1 control as remainder.
    • 5000 (10%) A
    • 5000 (10%) B
    • remainder (80%) C
{experiment name="AB-No-Champ" strategy="split" conversion="uniqueclicks"}
   {variant name="A" sample-size="5000"}Test A{/variant}
   {variant name="B" sample-size="5000"}Test B{/variant}
   {variant name="C"}Test C{/variant}
{/experiment}
  • Test 1 single variant (same size) and 1 control as remainder.
    • 5000 (10%) A
    • remainder (90%) B
{experiment name="AB-No-Champ" strategy="split" conversion="uniqueclicks"}
   {variant name="A" sample-size="5000"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment}

Split test with champion

Note: Not currently supported with messages scheduled to time zone of the recipient.
  • Test n variants and auto-send champion after x mins/hours.
    • 2500 sent A
    • 2500 sent B
    • Remainder sent to the champion after three hours.
{experiment name="Split-with-Champion" conversion="uniqueclicks" strategy="split-with-champion" send-champion="max-time" max-time="3h" sample-size="5000"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment} 
  • Test n variants where all variants are split by a specific percentage and auto-send champion after x mins/hours.
    • 10% A
    • 10% B
    • Remainder sent to the champion after two hours. 
{experiment name="Test" conversion="uniqueclicks" strategy="split-with-champion" send-champion="max-time" max-time="2h0m" sample-size-type="percentage" sample-size="20%"}
{variant name="A"}A{/variant}
{variant name="B"}B{/variant}
{/experiment}

Body experiment FAQs

Conditional logic with experiments

Experiments and Smarty run in parallel and asynchronously. Therefore it is not possible to use conditional logic to affect content in a message based on experiments.

For example, the following conditional code will not work as intended because the variable for $gender will always equal "boy" in this case:

{* will not work *}
{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
{variant name="A"}{$gender = "girl"}Hello girl,<br><br>{/variant}
{variant name="B"}{$gender = "boy"}Hello boy,<br><br>{/variant}
{/experiment}

{if $gender == "girl"}Girl content{elseif $gender == "boy"}Boy content{/if}

The recommended way to write this would be to put all desired content within each variant:

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Hello girl,<br><br>Girl content{/variant}
   {variant name="B"}Hello boy,<br><br>Boy content{/variant}
{/experiment}

You also have the option to place each content piece within an HTML content include. This improves usability when working with large amounts of code within the HTML editor.

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}{include 'content:GirlContent'}{/variant}
   {variant name="B"}{include 'content:BoyContent'}{/variant}
{/experiment}

 

In the next article, you can learn about Batch Experiment Performance

Comments

0 comments

Please sign in to leave a comment.