Exporting data from catalogs

You can export catalog records in the following ways:

• Export to a downloadable CSV, JSON or JSON lines file
  This method allows you to export catalog records to a file you can download and use as you wish. There are two options here:

  • by simple export
    Ideal for one-off, quick file extraction when you need the data just once.
  • in the Automation Hub
    Allows for cyclical, automated uploads of catalog record files. Best if you need the process to run regularly without manual intervention.

• Export to external services

  • You can export catalog records directly to external services that Synerise supports natively (for example, Amazon S3, Google Cloud Storage) using the Send File action nodes or to any endpoint using "Outgoing Integration" node.

  Benefits:

  • You get a draft workflow which you adjust to your needs.
  • This method lets you modify the file with catalog records by using the Data Transformation node in the workflow.

General prerequisites

You must have the permissions to export data.

Simple export

This method lets you perform a single export to a downloadable file.

1. Go to Data Modeling > Catalogs.
2. Select a catalog.
3. On the top bar, click Export.
   Result: A pop-up shows.
4. Select Export single file.
5. Click Next.
6. In the Name field, enter the name of the export. This will allow you to easily identify your export in the list of scheduled data export jobs.
7. Click Next.
8. Confirm by clicking Run export.

Where to find the file?

9. To download the exported file, go to Data Modeling > Exports.
10. On the left panel, select Catalogs.
11. Find the file on the list.
12. Click
13. Select **Download CSV, Download JSON or Download JSON lines.

Export through the Automation Hub

This export method lets you:

• export a downloadable file
• export a file to external services
  • Synerise offers easy integration with some services by the Send File node, for example, "Send File" node to Google Cloud Platform.
  • If you can’t find the service to which you want to export the file, use "Outgoing Integration" node.

Prerequisites for this method

• Verify if you need to modify the file with records before the export. If so, prepare a data transformation rule which you will later select in the settings of the workflow.
• Create a connection with the service to which you want to export the file with catalog records.
  If you can’t find the service to which you want to export the file, you can use "Outgoing Integration" node in the workflow and you can create the connection directly in the node configuration.

Configure and launch workflow

1. Go to Data Modeling > Catalogs.

2. Select a catalog.

3. On the top bar, click Export.
   Result: A pop-up shows.

4. Select Export in Automation.
   Result: The Where do you want to export file? dropdown shows.

5. From the Where do you want to export file? dropdown select:

   • if you want to export a downloadable file, select Save file.
   • if you want to export to external services, from the dropdown list, select one of the services.
   • if no option fits your business case, select any option: you will modify the draft workflow to your needs in further steps.
     Result: You are redirected to a draft workflow.
     
     Tip: If you were not redirected to the workflow, disable your browser’s pop-up blocker.

6. If you want to schedule the frequency and date and time when the workflow will trigger, edit the configuration of the "Scheduled Run" node.

7. In the Get Catalog Data node, leave the settings at default (the catalog is already selected).
   

   Note: You can read the node documentation in "Get Catalog Data" node.

8. In the Data Transformation node, select the transformation rule according to which the file with the records will be modified before export. If you don’t need the node, you can remove it.

9. Configure the node with the export:

   • If you selected Save File, go to the node settings and enter the name of the file to which catalog records will be exported.
   • If you selected an external service, go to the node settings:
     1. Select a connection.
     2. Fill out the configuration form.
        To get instructions to filling out the configuration form, find the service on the list and proceed to the node instructions.
        Tip: If the list doesn’t include the service to which you want to export, you can replace the node with the "Outgoing Integration" node - you can set up the connection and the endpoint to which you will make a request in the node directly.

10. In the upper-right corner, click Save & Run.
    Result: The workflow will be active and the actions in the workflow will be triggered as defined in the Scheduled Run node. The logs of operations in the workflow are available:

    • for the data transformation: enter the configuration of the workflow ( Automation Hub > Workflows), select the Transformation logs tab.
    • for the export, go to Data Modeling > Exports, on the left pane, select Automation.