Using Full Message Experiments in Cordial

Full Message Experiments help you understand what’s truly working in your batch email campaigns, by comparing variants and measuring statistically meaningful results.

By creating a Full Message Experiment report you can quickly generate results for any sent batch messages that use Cordial’s Random Number Generator (RNG) method for audience splits. This lets you easily analyze performance and identify winning variants without manual spreadsheets or offline analysis. 

Key benefits include:

• Centralized insights: Access all results in one dashboard rather than offline reports.
• Better accuracy: Results aggregate across messages to improve statistical reliability.
• Faster learning: Quickly identify which messages and AI features drive stronger engagement or revenue.

Contact your CSM if you aren't sure if your account is equipped with RNG audiences

What Are Full Message Experiments?

Full Message Experiments are designed to help you run and analyze experiments across entire messages, not just subject lines or single components. They include several key layers of organization and visibility.

1. Full Message Experiments are the container that groups together multiple batch messages and provides an overall performance summary.
2. Test Groups are the sets of tests that include variants from a single send instance. These are commonly the Control and Challenger variants of a send on a single day. Multiple test groups can be added to one experiment.
3. Messages are the variants within each test group that use RNG-split audiences and are intended to be grouped together for analysis.

Example

Use Experiment Reporting to track an ongoing test of AI-driven product recommendations across five batch email sends. Add each send to the experiment, designate your control and test variants, and view aggregated results, including click rate, order rate, and revenue per email, to determine whether the test variant is producing a statistically significant lift over time.

Before you begin

Before using Experiment Reporting, confirm the following:

• Your account has Experiment Reporting enabled. This feature is available to select accounts. Contact your Client Success Manager or submit a support request to request access.
• You have already sent the batch email messages you want to analyze.
• Your messages used Cordial's random number generation method to split your audience into control and test groups. Experiment Reporting does not automate audience splitting, your send setup must have handled this before the messages were sent.
• You know which messages served as the control variant and which served as the test variant.

Creating a Full Message Experiment Report

Adding RNG‑enabled batch emails to an experiment report and generating results is straightforward.

1. Navigate to Analytics -> Full Message Experiments
2. Select Create Experiment Report
    

3. Name your experiment, add a description and select the Primary test metric. 
    

   

    

4. Enter a new label for your test variants or use the default labels of Control and Challenger
    

5. Create your first test group and select the batch messages that correspond to each. Optionally, add multiple test groups and sets of messages.
    

6. An aggregate performance report is immediately generated.

Review experiment results

After adding your messages, the experiment performance page aggregates results across all message groups and displays the following:

• Key metrics: open rate, click rate, order rate, revenue per email, unsubscribe rate, bounce rate, and delivered rate, shown per variant.
• Relative lift: the performance difference between your test and control variants across each metric.
• Statistical winner: which variant is performing better, based on the volume and results of the sends included in the experiment.
   

The accuracy and reliability of your results depend on how your audience splits were set up prior to sending. Experiment Reporting analyzes the data from your existing sends, it does not validate or enforce experiment methodology during setup.