Overview

A subject line experiment allows you to test multiple subject lines in a single message to optimize the following conversion metrics:

Experiment types

There are two experiment options available:

• Split test
  • Split your audience evenly per variant.
  • Set the variant sample size (in percentage or number) and allow for a control variant.
• Split test with champion: Set sample size per variant (in percentage or number) and automatically send a champion.

Enable the subject line experiment

1. Log in to Cordial and navigate to Messages > Create New Message or open a draft message.

2. Select Edit in the Message Header pane.

3. Enable the subject line experiment by turning the the toggle switch to On.

4. Give your experiment a name. Names can only contain alphanumeric characters, dashes, underscores and no more than 30 characters (i.e. Holiday_Subject_Test).

AI-generated subject lines

You can utilize AI-generated experiment subject lines to optimize the impact of your messaging. Generative AI is available for experiment variants and controls so you can analyze how people react to different subject line variations.

1. Click Generate AI Content in the Subject experiment pane.

3. From within the generative AI pane, enter a prompt and click Generate AI Content. Multiple variations of usable content will appear.

4. To customize the text, select one or multiple of the buttons: Shorten, Lengthen, Urgent, Casual, Funny.

To keep your subject lines concise and effective, AI content in this pane won't exceed one line.

5. Click Use Content under the AI-generated text you wish to use, and it will appear in your message subject line.

Choose your conversion metric

Choose what conversion metric you’d like to optimize for. This is the metric that will be used to determine which variant performs better.

• You can optimize based on a contact's message engagement (unique opens and unique clicks), orders (orders to open and orders to click), or revenue (revenue to open and revenue to click). 
• To take advantage of order and revenue conversion metrics, you need to pass order and revenue data to your account via the API or JavaScript listeners.

Select experiment strategy

Select your Experiment strategy: Spilt test or Split test with champion.

Split test

A split test will determine how the audience is split for the experiment.

• Split audience evenly per variant: Splits the audience evenly between the variants.
• Allow a control variant: Allows you to add a control variant in addition to the experiment variant(s).
  • You can choose the split based on a percentage or a number.

Allowing for a control variant in the Split audience dropdown lets you set aside a group of people (control) to test your variant against. Once added, you'll see the control variant input show up below the variants.

You can set the test size as a percentage or a specific number of contacts.

Split test with champion

When the strategy is set for Split test with champion, you can have the remainder of the audience applied to the best performing variant after a specified delay.

You can set the test size as a percentage or a specific number.

Configure variants

1. Name each of your variants (note: only alphanumeric characters, dashes, underscores).

2. Enter the subject lines to test.

3. Click + Subject Variant to add more variants to test.

4. Once finished, click Save. You'll see a preview of the details in the message header.

Subject line experiment scenarios

1. Split test without champion

Test n variants where all variants are evenly split across the audience

• 50% A
• 50% B

Test n variants where all variants are evenly split across the audience

• 33.3% A
• 33.3% B
• 33.3% C

Test n test variants (same size population) and one control as remainder

• 600 variant A
• 600 variant B
• Remainder control

Test n test variants (as a percentage) and one control as remainder

• 40% variant A
• 40% variant B
• Remainder control (20%)

Test one single variant and one control as remainder

• 1,000 variant A
• Remainder control

2. Split test with champion

This feature is not currently supported with messages scheduled to time zone of the recipient.

Test n variants and auto-send champion after x mins/hours

• 500 sent A
• 500 sent B
• Remainder sent to champion after three hours

Test n variants and auto-send champion after x mins/hours

• 20% sent A
• 20% sent B
• Remainder sent to champion after three hours (60%)

In the next article, you can learn about body experiments.