Transformations in the Fivetran Dashboard

Learn how to manage Transformations for dbt Core* in your dashboard.

NOTE: For more detailed information about Quickstart data models, see the Manage data models section of our Transformations docs.

View transformations

To view transformations for a specific destination, click Transformations in the left navigation pane and choose your destination. On the Transformations page, you can see the status and time of each transformation’s last run, the run frequency, the transformation type, and the connectors associated with that output model.

You can filter the transformations list by the following statuses:

• Succeeded - your latest transformation run is successfully completed
• Failed - your latest transformation run has failed
• Scheduling - your transformation is being started at the moment
• Running - your transformation is currently running
• Blocked - your transformation is blocked by the failure of one or more upstream tasks
• Pending - your transformation has been created but has not completed its first run yet
• Paused - your transformation has been created but is disabled, which means it not running on any schedule

To view information about an individual transformation, click the output model name. Each transformation profile includes the following tabs:

• Run log: View active and past runs
• Job commands: View dbt job commands of Transformations for dbt Core* and edit dbt job commands of Scheduled in Fivetran transformations
• Data models: Manage the data models included in you Quickstart data model package
• Schedule: Manage current schedule settings
• Details: View details about the transformation and delete it

You can enable and disable the transformation using the toggle in the top-left corner of the Transformation details page. You can also manually run the transformation anytime by clicking Run. To run the transformation with full_refresh, click the arrow beside Run and then click Run with full refresh.

Run log

On the Run log tab, you can view your transformation’s recent run history and the currently running transformation, as well as the start time, duration, and status of each model run. Click the relevant run's row to see the run details.

Job commands

On the Job commands tab, you can view your transformation's job commands. For Scheduled in Fivetran transformations, you can manage job commands in your Fivetran dashboard:

• Add job commands
• Edit job commands
• Delete job commands
• Change the order of their execution

Data models

On the Data models tab, you can manage the output models included in your Quickstart data model package:

• You can see each model's documentation by clicking the dbt Docs link
• You can use the Included in run toggle to include data models in each run of the Quickstart transformation or to exclude them
• You can see the list of Unavailable models. To make them available, you must sync the tables and columns required for these models.

Schedule

On the Schedule tab, you can view and edit your output model’s current schedule settings. There are the following schedule options:

• Integrated: Fivetran will run your transformation after each upstream connector is synced. You can see the list of Upstream Connectors and Sync Frequency.

• Custom: Fivetran will run your transformation at the frequency of your choice. If you enabled the Smart syncing option, the model won't run until connectors have completed syncing when schedules overlap. Custom scheduling allows you to select particular days of the week to run your transformation, a run frequency within a day, and, when the run frequency is set to 24 hours, a particular run time for your transformation. Available run intervals are as follows:

  • 1 min
  • 5 min
  • 15 min
  • 30 min
  • 1 hour
  • 2 hours
  • 3 hours
  • 6 hours
  • 8 hours
  • 12 hours
  • 24 hours

  When you set 24 hours as a run interval, you can also select a particular time to run your transformation at.

• Cron: Fivetran will run your transformation based on the specified cron schedule. You can define and use multiple cron schedules to accommodate various transformation timings.

You can also access the Schedule tab from the main Transformations page. Click the three dots on the right side of the row, then select View schedule.

To edit the schedule, follow the instructions in the edit transformation schedule section.

Details

On the Details tab, you can view the following information about your transformation:

• Transformation ID: A unique ID that Fivetran assigns your transformation
• Created on: When the transformation was created
• Created by: Who created the transformation
• Average run time: The average duration of the transformation’s runs
• Last edited on: When the transformation was last edited within Fivetran
• Last edited by: The user who last edited the transformation within Fivetran
• Target: The target used for the transformation

On the Details tab, you can also remove a transformation. Learn how in the delete transformations section.

Edit dbt Core connection settings

To view or edit your destination’s dbt Core connection settings, follow these steps:

1. On the Transformations main page, click Settings.
2. Go to the dbt Core tab.
3. Click Edit connection details.
4. Edit your dbt Core connection settings, then click Save & Test.

   TIP: When editing the connection settings on the dbt Core tab, you can click the blue hints to navigate to the relevant sections of the embedded Setup Guide displayed on the right.

NOTE: If you change your repository, we connect to the new repository and remove all transformations associated with the original repository.

Disconnecting Git repository containing your dbt project

On the dbt Core tab, under Settings, Click to disconnect the git repository containing your dbt project.

Add transformations

To add a transformation, follow these steps:

1. Click Transformations.

2. Type the destination name in the Search by name field to find the destination, or select the destination from the list.

3. Click Add transformation.

4. Depending on the transformation type you want to add, do the following:

   • For Quickstart data models, click Add data models.
   • For other transformation types, if the project exists, click Create Job.

5. Define the settings for the selected transformation type.

   • Quickstart data models - perform the following steps:

     i. Select the Source type.

     ii. Select Connector.

     iii. Check Use a custom schema checkbox and enter the name of the Custom schema if you want to use it.

     NOTE: If you use a custom schema, any data processed by this transformation will be placed within that schema in your data destination.

     iv. (Optional) You can Read model descriptions in the embedded documentation for the Output models.

   • dbt Core - perform the following steps:

     i. Select target.

     ii. Enter Job name.

     iii. Under Enter dbt command, enter the name of the command and the command itself.

     iv. (Optional) Add another dbt job command by clicking +Add command and providing the command name and command itself.

6. Set schedule by selecting either of the following schedule types:

   • Integrated - this schedule type is not available for transformations for dbt Core until you add a connector to your transformations
   • Custom - if you opt to use Smart Syncing, this schedule type is not available for transformations for dbt Core until you add a connector to your transformations
   • Cron

7. Click Save.

Manage job commands

Add job command

To add a job command, follow these steps:

1. Go to the transformation’s Job commands tab.
2. Click +Add command.
3. Enter the name of the command.
4. Enter the command.
5. Click Save.

Edit job command

To edit a job command, follow these steps:

1. Go to the transformation’s Job commands tab.
2. Click +Add command.
3. (Optional) Enter the new name of the command.
4. (optional) Enter the new command.
5. Click Save.

Change job command order

To change job command order, follow these steps:

1. Go to the transformation’s Job commands tab.
2. Use arrow buttons beside job commands to change the job command order.
3. Click Save.

Delete job command

To delete a job command, follow these steps:

1. Go to the transformation’s Job commands tab.
2. Click the trash bin icon beside the job command you want to delete.
3. Click Save.

Edit transformation schedule

NOTE: You can only edit a transformation's schedule and Scheduled in Fivetran transformation's job commands in the Fivetran dashboard. If you want to change the actual SQL statements within the transformation, you must do that in your dbt project.

To edit a transformation's schedule, follow these steps:

1. Go to the transformation’s Schedule tab.
2. Edit the schedule settings.
3. (Optional) You can also select the Execute tests after each model run checkbox.
4. Click Save.

Delete transformations

IMPORTANT: When you delete a transformation, the output model is completely removed from your dashboard, and you can no longer view its run history. If you want to schedule this output model later, you must add a new transformation. The new transformation will not include run history from the original model.

To delete a transformation from the Fivetran dashboard, follow these steps:

1. Go to the transformation’s Details tab.
2. Click Remove transformation.
3. A pop-up window appears with a warning about deleting your transformation. Read it, then click Remove.

You can also remove a transformation on the Transformations main page. Click the three dots on the right side of the row, then click Delete Transformation.

Transformation bulk actions

Bulk actions allow you to select multiple transformations and perform any of the following actions at once:

• Pause selected transformations - only available for Transformation for dbt Core* and Quickstart transformations
• Resume selected transformations - only available for Transformation for dbt Core and Quickstart transformations
• Edit schedule of selected transformations - only available for Transformation for dbt Core - Scheduled in Fivetran and Quickstart transformations
• Remove selected transformations

When you select the relevant transformations on the Transformations list page by checking the box beside the transformation name, the bulk action pane appears at the bottom of the transformation list.

It displays the number of the selected transformations along with the relevant bulk action buttons. You can filter the list to only display the selected transformations by clicking Show Selected.

Pause selected transformations

To pause transformations using the bulk action mode:

1. Select the transformations you want to pause by checking the box beside their names.
2. Click Pause.

Resume selected transformations

To resume transformations using the bulk action mode:

1. Select the transformations you want to resume by checking the box beside their names.
2. Click Resume.

Edit schedule of selected transformations

To edit the schedule of transformations using the bulk action mode:

1. Select the transformations you want to change the schedule for by checking the box beside their names.
2. Click Edit schedule.
3. Edit the schedule settings.
4. Click Save.

Remove selected transformations

To remove transformations using the bulk action mode:

1. Select the transformations you want to remove by checking the box beside their names.
2. Click Remove.
3. In the confirmation pop-up window, click Remove.

Troubleshoot failed transformations

To troubleshoot a failed transformation, use the Run log tab:

1. Click the relevant failed run in the Run log tab.
2. Read the run log to find out why your transformation failed.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.