How can we help?

Data Jobs and Transformations overview

Overview

Data Jobs enable you to leverage customer data in several powerful ways. You can supply data from your Cordial account or from an external source and perform any number of functions using that data. You can also perform advanced transformations on your data for specific use cases.

The Cordial user interface (UI) streamlines these processes into simple workflows, offering no-code solutions.

We also provide a flexible scripting interface for running advanced Data Transformations. You can create custom scripts to perform one or more data operations in unison. Your script can contain Smarty data transformation logic utilizing the utilities you've come to know and love. Additionally, Cordial has utilities specific to Data Transformations that allow you to operate on almost any data collection in Cordial.

Available Data Jobs

You can use Data Jobs in Cordial to 

Create a Data Job

1. Log in to Cordial and navigate to Data Jobs > Create New Data Job to get started.

2. Select one of the available Data Jobs from the menu.

3. After selecting the type of Data Job you want to create, give the job a unique name and choose the Job Occurrence—one-time or recurring—and Priority—low, standard, or high.

  • One time: Run a data job once only.
  • Recurring: Used to run data jobs automatically based on a set schedule.

4. Configure your data job on the Data Job creation page. You can learn more about the process for each data job type in the following articles:

View status of Data Jobs

You can view your draft, one-time, and automated Data Jobs using the left navigation menu.

Data Jobs > One Time Data Job Drafts: View a list of one time Data Jobs in draft status. A draft is saved immediately after you create a new one time data job.

Data Jobs > Completed One Time Data Jobs: Displays a list of one time Data Jobs that you ran. Depending on the Data Job complexity and the number of concurrently requested jobs, the status may be one of the following:

  • Complete: The Data Job is done running.
  • Pending: Waiting to begin running.
  • Failed: Data Job stopped running before it was completed. Please check the log for more information.

Status labels that indicate Data Job is still running:

  • Processing
  • Pre-processing
  • RecordsQueued

Data Jobs > Data Job Automations: View the status of created Data Job automations. Job automations are added to this list immediately after you create them, including those that have yet to be enabled and published.

Data Transformations

You can perform Data Transformations to update any of the incoming data and perform other complex transformation functions before sending data to its destination.

1. After selecting Create New Data Job > Transformations, choose from the following three data transformation options.

  • Custom: create a data transformation from scratch to fit your specific use case.
  • Scripted export: create a scripted export of your Cordial data to utilize on another platform, or however you see fit.
  • Scripted import: create a scripted import of your Cordial data to use data from external sources.

2. Configure your Data Source and Data Mapping.

3. Click Edit under Data Transformation to target and store specific data points of the larger data set into one or more internal destinations (see the following section).

  • Pre Job Script: Runs Smarty only once and is used to set variables and other data objects for use by the main script.
  • Transformation Script: Smarty in the main transformation script will run repeatedly for each row of data.
  • Post Job Script: Run Smarty only once after the main transformation script is done rendering. 

Use case

See the Swap supplements: updating and inserting large data sets use case.

4. Click </> to paste in content from your HTML Content Library.

5. Click Show Column Headers to see a list of pre-built variables corresponding to the header row of the CSV file.

Test your script

To test a transformation, you must provide a sample row of data in order to see the expected results and test the validity of your script. During Data Job setup that contains a transformation, click Edit to access the Transformation Script window.

All Smarty utilities in the transformation will execute and applicable data updates/inserts will be performed, so we recommend using contact profiles reserved for testing purposes.

If your data source is internal contacts, you can select an existing contact to test the transformation, but remember that running the test script will add or update the selected contact record. We recommend running your test with a contact record reserved for testing purposes.

You can select the desired test contact from the bottom left of the Transformation Script window via the Test as menu.

If your data source is internal events, orders, or supplements, you can enter a sample record written in JSON. To display the JSON input dialog window, click the Test Script button in the bottom left of the Transformation Script window.

If your data source is an external CSV or JSONL file, you will be able to insert sample values of a record or insert a row directly from your file. To display the import file test options dialog window, click the Test Script button from the bottom left of the Transformation Script window.

Click the Show Column Headers button at the bottom left of the Transformation Script window to see a list of pre-built variables corresponding to the header row of the CSV file.

Status log and rendered output

In your data transformation, you can use the Log tab to see a breakdown of utility functions that have been executed with success or failure. Click on individual rows to expand for additional details.

We recommend using the $utils->jsonPrettyPrint($object) utility to output values as you develop your script. The utility will render data objects in the Rendered Output tab.

The script will stop as soon as a failure is encountered. Any data updates performed before the failure will not be reverted. No additional utilities will be executed when a failure is encountered.

Supported Smarty utilities

A collection of Smarty utilities are available for use with data transformations. For a complete list of supported utilities, visit the Cordial utilities reference page.

API error responses

The Cordial API will return an error object with an errorKey and message if there's a problem with an API call. Below is a list of errors specific to the data automations (Data Jobs) API endpoint, along with suggested modifications to resolve each error. If you receive an error from this API endpoint that is not listed in this table, it is likely recorded within the Global API error responses page.

errorKey Message Modifications
DATA_JOBS_TRANSFORMATION_FEATURE_IS_DISABLED Current account does not support transformations through API. Current account does not support transformations through API.
DATA_JOBS_MISSING_TRANSFORMATION_RECORDS Transform record(s) should be provided. Ensure the correct transformation records are provided.
DATA_JOBS_AUTOMATION_KEY_NOT_FOUND Data Automation does not exist. The provided key did not return a Data Automation.
DATA_JOBS_API_TRIGGER_IS_DISABLED Trigger API is disabled. The trigger is disabled.

Comments

0 comments

Please sign in to leave a comment.