How to Use Fivetran's Sample dbt deployment.yml Filelink
NOTE: This article applies to legacy transformations with dbt Core.
Use Caselink
You want to use the deployment.yml
file to perform advanced orchestrations with your dbt Git repository. You want to do the following:
- Run specific models
- Run specific tasks
- Run other dbt commands
Environmentlink
dbt Git repository and your local source code editor
Recommendationlink
Fivetran provides a sample deployment.yml file that enables us to run your dbt commands on a schedule. You can edit the jobs in the deployment.yml
file to reflect how and when you want Fivetran to run your dbt commands.
The deployment.yml
file allows you to define multiple jobs, each of which translates to a transformation in your Fivetran dashboard.
Each job include three main components:
- Name: Enter a unique job name. The name is displayed in your Fivetran dashboard once your jobs are imported.
- Schedule: Define when this job should run using the Cron format. It is possible to define a list of Cron schedules using a YAML array. Both the single line and multiline formats are supported.
schedule: ["0 12 * * *", "0 5 * * *"] # Quotes needed to shield cron expression
schedule:
- 0 12 * * *
- 0 5 * * *
Steps: A job performs multiple steps sequentially, so define each step here:
- Name: Give each step in your job a name. This enables you to track the steps in the logs.
- Command: Enter the dbt command that should run in this step.
Common workflowslink
To ensure data integrity, use two steps (run the models and test the models) within a job:
- name: daily
schedule: 0 12 * * * # This example will run every day at 12:00pm
steps:
- name: run models
command: dbt run
- name: test models
command: dbt test
You can also run the models in a specific order by including steps that only run certain models:
- name: daily
schedule: 0 12 * * * # This example will run every day at 12:00pm
steps:
- name: run first sales model
command: dbt run --models sales_model_1
- name: run second sales model
command: dbt run --models sales_model_2