Overview

In addition to subject line experiments, you can build multi-variant message body experiments using Sculpt Experiment Blocks or the HTML Editor.

The ability to add a control variant for message body experiments is only available for batch messages.

Body experiments support the same conversion metrics as subject line experiments:

• Unique Opens
• Unique Clicks
• Revenue Per Send
• Orders Per Send
• Orders Per Click
• Revenue Per Click
• Uniqe Clicks to Unique Opens

Multiple experiments can run concurrently in batch messages, but they must use the same testing strategy and populations (ex: strategy="split" or strategy="split-with-champion"). If you set your subject line experiment to split with champion, your body experiment must also use the same strategy and populations.

Create body experiments using Sculpt Blocks

Sculpt makes it simple to add multiple body experiments to a message using Experiment Blocks.

1. To add an Experiment Block to a Sculpt Template (which you use to create Sculpt messages), navigate to Content > Sculpt Templates > New or select an existing Template.

2. Within the Template, select the Preview in Sculpt message editor toggle and click the Block Library button.

3. The Experiment Block will show up as the first Block in the Block Library. Click and drag the Experiment Block into your Sculpt Template.

2. Once you've added the Experiment Block to your Template, click the Experiment Block to configure its settings.

• Experiment name: Add a name for the experiment (underscores will automatically replace spaces in the name).
• Optimize: Choose a metric to optimize
  • Unique Clicks, Unique Opens, Revenue Per Send, Orders Per Send, Orders Per Click, Revenue Per Click, Unique Clicks to Unique Opens
• Experiment strategy: Choose a strategy
  • Batch messages: Split or Split with champion
  • Automated messages: Multi-armed bandit
• Variants: Add variants to the experiment as desired.

3. After you've configured the experiment settings and added the desired variants, you can drag Blocks into each variant using the variant dropdown button (highlighted above).

4. You can preview experiments using the variant dropdown in the Experiment Block, or by using the Experiment Preview button and choosing the desired variants.

• You can learn more about creating Sculpt messages here.

Create body experiments in the HTML Editor

You can also build message body experiments using Cordial's template syntax (Smarty) in the HTML Editor of your message.

1. For new messages, navigate to Messages > Create New Message and select HTML Editor.

• To add a body experiment to an existing message, navigate to Messages > Drafts and select the message.

2. Click Edit under Message Content to access the HTML Editor.

3. Create your body experiment using Smarty. You can get started with the example experiment configurations below.

For help creating advanced body experiments using Smarty, contact your CSM.

4. Once you've created your body experiment using Smarty, you can view the experiment variants by clicking the Experiments button.

Body experiment performance

1. View the performance for your body experiment under Message Performance > Experiments.

2. Under Test Variants, select which variant to view. You can also choose to view Performance at time of champion selection or the Current performance for the experiment.

3. Click the Export button at the top to export your experiment results.

Example Smarty experiments

Split test without champion

The experiment element in batch messages

The variant element

• Test n variants where all variants are evenly split across the audience
• 50% A
• 50% B

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment}

• Test n variants where all variants are evenly split across the audience
• 33.3% A
• 33.3% B
• 33.3% C

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
   {variant name="C"}Test C{/variant}
{/experiment}

• Test n variants (same size) and 1 control as remainder
  • 5000 (10%) A
  • 5000 (10%) B
  • remainder (80%) C

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueclicks"}
   {variant name="A" sample-size="5000"}Test A{/variant}
   {variant name="B" sample-size="5000"}Test B{/variant}
   {variant name="C"}Test C{/variant}
{/experiment}

• Test 1 single variant (same size) and 1 control as remainder
• 5000 (10%) A
• remainder (90%) B

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueclicks"}
   {variant name="A" sample-size="5000"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment}

Split test with champion

Not currently supported with messages scheduled to time zone of the recipient.

• Test n variants and auto-send champion after x mins/hours
• 2500 sent A
• 2500 sent B
• Remainder sent to the champion after three hours.

{experiment name="Split-with-Champion" conversion="uniqueclicks" strategy="split-with-champion" send-champion="max-time" max-time="3h" sample-size="5000"}
   {variant name="A"}Test A{/variant}
   {variant name="B"}Test B{/variant}
{/experiment} 

• Test n variants where all variants are split by a specific percentage and auto-send champion after x mins/hours
  • 10% A
  • 10% B
  • Remainder sent to the champion after two hours. 

{experiment name="Test" conversion="uniqueclicks" strategy="split-with-champion" send-champion="max-time" max-time="2h0m" sample-size-type="percentage" sample-size="20%"}
   {variant name="A"}A{/variant}
   {variant name="B"}B{/variant}
{/experiment}

Body experiment FAQs

Conditional logic with experiments

Experiments and Smarty run in parallel and asynchronously, therefore it is not possible to use conditional logic to affect content in a message based on experiments. That said, we have developed workaround solutions.

For example, the following conditional code will not work as intended because the variable for $gender will always equal boy in this case:

{* will not work *}
{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
{variant name="A"}{$gender = "girl"}Hello girl,<br><br>{/variant}
   {variant name="B"}{$gender = "boy"}Hello boy,<br><br>{/variant}
{/experiment}

{if $gender == "girl"}Girl content{elseif $gender == "boy"}Boy content{/if}

The recommended way to write this would be to put all desired content within each variant:

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}Hello girl,<br><br>Girl content{/variant}
   {variant name="B"}Hello boy,<br><br>Boy content{/variant}
{/experiment}

You can also place each content piece within an HTML content include. This improves usability when working with large amounts of code within the HTML editor.

{experiment name="AB-No-Champ" strategy="split" conversion="uniqueopens"}
   {variant name="A"}{include 'content:GirlContent'}{/variant}
   {variant name="B"}{include 'content:BoyContent'}{/variant}
{/experiment}