Overview

A subject line experiment allows you to test multiple subject lines in a single message to optimize the following conversion metrics:

Experiment types

There are two experiment options available:

• Split test
  • Split your audience evenly per variant.
  • Set the variant sample size (in percentage or number) and allow for a control variant.
• Split test with champion: Set sample size per variant (in percentage or number) and automatically send a champion.

Enable the subject line experiment

1. Log in to Cordial and navigate to Messages > Create New Message or open a draft message.

2. Select Edit in the Message Header pane.

3. Enable the subject line experiment by turning the the toggle switch to On.

4. Give your experiment a name. Names can only contain alphanumeric characters, dashes, underscores and no more than 30 characters (i.e. Holiday_Subject_Test).

Choose your conversion metric

Choose what conversion metric you’d like to optimize for. This is the metric that will be used to determine which variant performs better.

• You can optimize based on a contact's message engagement (unique opens and unique clicks), orders (orders to open and orders to click), or revenue (revenue to open and revenue to click). 
• To take advantage of order and revenue conversion metrics, you need to pass order and revenue data to your account via the API or JavaScript listeners.

Select experiment strategy

Select your Experiment strategy: Spilt test or Split test with champion.

Split test

A split test will determine how the audience is split for the experiment.

• Split audience evenly per variant: Splits the audience evenly between the variants.
• Allow a control variant: Allows you to add a control variant in addition to the experiment variant(s).
  • You can choose the split based on a percentage or a number.

Allowing for a control variant in the Split audience dropdown lets you set aside a group of people (control) to test your variant against. Once added, you'll see the control variant input show up below the variants.

You can set the test size as a percentage or a specific number of contacts.

Split test with champion

When the strategy is set for Split test with champion, you can have the remainder of the audience applied to the best performing variant after a specified delay.

You can set the test size as a percentage or a specific number.

Configure variants

1. Name each of your variants (note: only alphanumeric characters, dashes, underscores).

2. Enter the subject lines to test.

3. Click + Subject Variant to add more variants to test.

4. Once finished, click Save. You'll see a preview of the details in the message header.

Subject line experiment scenarios

Split test without champion

Test n variants where all variants are evenly split across the audience.

• 50% A
• 50% B

Test n variants where all variants are evenly split across the audience.

• 33.3% A
• 33.3% B
• 33.3% C

Test n test variants (same size population) and 1 control as remainder.

• 1200 Variant A
• 1200 Variant B
• Remainder control

Test n test variants (as a percentage) and 1 control as remainder.

• 40% Variant A
• 40% Variant B
• Remainder control (20%)

Test 1 single variant and 1 control as remainder.

• 1500 Variant A
• Remainder control

Split test with champion

This feature is not currently supported with messages scheduled to time zone of the recipient.

Test n variants and auto-send champion after x mins/hours.

• 500 sent A
• 500 sent B
• Remainder sent to the champion after 3 hours

Test n variants and auto-send champion after x mins/hours.

• 20% sent A
• 20% sent B
• Remainder sent to the champion after 3 hours (60%)

In the next article, you can learn about body experiments.