"Data Transformation" node

The Data Transformation node is useful for the workflows whose purpose is to retrieve the data, structure it to meet requirements and process them further. Examples of such workflows can be retrieving a file from an external source and importing it to Synerise. The intermediate step between these two actions is converting the data structure, which can be achieved by means of the Data Transformation node.

The Data Transformation node performs the operations on a file according to the logic described by a transformation rule which you must select in the configuration of this node.

How does it work

When you create a workflow which includes the Data Transformation node, you must add a node that contains the target file to be transformed (see Prerequisites) right before the Data Transformation node.

When a launched workflow arrives at the Data Transformation node, the system performs the modifications to the target file according to the rules defined in the transformation rule selected in the node. It is important to select the transformation rule designated for the structure of the target file for the successful transformation.

What if the target file does not fully comply with the transformation rules?

Transformation will be performed but may not always be successful. The Data Transformation node accepts uneven data (for example, when a file contains a column that keeps information about the resolution of a product, which is required for items such as computers but not necessary for AGD equipment) but may not always transform them correctly.

Example 1: If a transformation rule was based on a file sample with columns: A, B, and C. The Data Transformation node receives the actual file with only B and C columns, the transformation of such file will be performed.

Example 2: A transformation rule was based on a file sample with columns: A, B, and C, but the Data Transformation node receives the actual file with A, B and D columns. If the transformation rule contains a condition a column D qualifies to (for example, a specific value is added to columns whose names start with A, B, C, and D), then the transformation will be performed, even if the D column wasn’t present in a file sample.
However, if column D doesn’t qualify for any condition, the column will be imported as delivered in the file.

Prerequisites

• You must create a transformation rule.
• The workflow which includes the Data Transformation node must contain one of the following nodes:
  • Local File
  • Get File (HTTPS)
  • Get File (SFTP)
  • Get File (Google Cloud Storage)
  • Get File (Azure Blob Storage)
  • Get file (Amazon S3)
  • Get Profiles

Node configuration

1. Click the Data Transformation node.
2. From the dropdown list, select the transformation rule.
   Result: The pop-up window displays the column a file contains.

Example of use

This example describes a procedure of creating a workflow that performs a single import of a file with transactions uploaded from the device. The workflow performs the transformation of the file with transactions so it complies with the Synerise requirements. After the transformation, the file with transactions is imported to Synerise.

1. Learn about the requirements for import of transactional data.
2. Prepare a transformation rule according to the requirements.
3. Create the new workflow and start it with the Scheduled Run node.
4. In the configuration of the node, set it to be scheduled immediately.
5. Add the Local File node. In the configuration of the node, upload the file with transactions.
   
   Note: If you store the file on a server, select HTTP or SFTP node and follow the instructions on the interface.
6. Add the Data Transformation node.
7. In the configuration of the node, select the transformation you prepared in step 2.
8. Add the Import Transactions node. In the settings of the node, you can check the list of the required columns.
9. Add the End node.

   

10. Click Save & Run.