Workflow

Introduction

Workflow is one of the most important features of WitCloud. The task is to start the previously configured modules automatically in the correct order. This ensures that both the inputs used for reporting and the report tables on the basis of which the visualizations are created are up-to-date.

How to create a workflow

The workflow is created in two ways:

• automatically - upon module creation, the system automatically generates the corresponding workflow that will update the data.
• manually - in custom cases, it is possible to create a workflow using a form.

Automatic workflow

When creating a Collect module, the system automatically creates several workflows for us. Their tasks are as follows:

One-time collection of historical data. Regular execution of the data update process for the current day. Daily execution of a data reconciliation task for the past days, depending on the system requirements, to collect potentially lost data. Upon completing the configuration of a connector, we receive a summary indicating that a data refresh schedule has been created for us, which means that automatic workflows have been created on our behalf.

The created workflows can be found in the workflow tab on the left-hand sidebar. Then select Auto-generated.

Creating a Workflow module "manually"

In some more advanced cases, it may be necessary to create a workflow manually. This situation may occur, for example, if we need a specific sequence of operations or if we are creating a more advanced report based on indirectly calculated data.

Configuration

To define a new task, select workflow from the left panel.

Click Add new Workflow.

First, enter a name for the workflow in the Enter Workflow name field.

In the second step, add modules. You can do this by clicking the "➕" icon or using the task/group task function. A Task is performed for a single module, while a Group Task combines modules into a group. This is helpful when we want to run several different modules for the same date range.

After adding a task, several settings will appear that need to be filled in.

1. First, use the Select Resource button to choose the tasks to be performed (collect, process, or report).

2. By clicking the calendar icon, you can specify the date range for task execution. The following options are available:

   1. Last X Days - dynamically determines dates from the current date to X days ago.
   2. Dynamic Date Range - dynamically determines dates. Allows setting a time range relative to the current date, e.g., from 14 days ago to 1 day ago.
   3. Static Date Range - allows specifying a fixed date range.
   4. Specified Date Range - specific dates listed separated by commas.

3. Clicking the settings icon allows you to choose whether the task should be retried in case of failure.

4. The trash icon button allows you to delete the added task.

When the Schedule Workflow option is disabled, the workflow can only be manually triggered from the list.

Upon completing the workflow creation, define a schedule for its execution by enabling the Schedule Workflow option. It is necessary to select a time zone in which the tasks will be carried out. It is possible to schedule the workflow at specific intervals (every X minutes, hours, days, weeks, or months).

After completing the configuration, click the Finish button.

Order of Actions in Workflow

Example of action

If we have a report based on data from multiple collectors, we place them all in the order before it. Below is an example Workflow:

1. Retrieve user behavior data from Google Analytics 4.
2. Retrieve cost data from Facebook Ads and Google Ads.
3. Retrieve data from Google Search Console.
4. Retrieve Ecommerce data from the e-commerce platform.
5. Run a report that combines all the collected data above.