dbt Transformations Beta
Fivetran integrates with Fishtown Analytics' data build tool (dbt) to power complex transformations. Fivetran connects directly to your Git provider and runs your dbt transformations according to your schedule.
To see how to orchestrate dbt transformations with Fivetran, watch this short video:
dbt is an open-source software that enables you to perform sophisticated data transformations in your destination using simple SQL statements. With dbt, you can:
- Write and test SQL transformations
- Use version control with your transformations
- Create and share documentation about your dbt transformations
- View data lineage graphs
How does dbt work?
Once you have set up dbt, you write SQL SELECT statements to transform your data. dbt runs these statements (a.k.a. "dbt models") in your destination and turns them into tables and views.
To work with dbt, you can either use the dbt Cloud application or the dbt Command Line Interface (CLI). Fivetran can run dbt models created with either dbt Cloud or dbt CLI.
dbt Cloud is a hosted service that helps you manage your dbt transformations. dbt Cloud provides support for scheduling and monitoring your dbt models, CI/CD, and generating documentation. It also provides an Integrated Developer Environment where you can develop, run, and version control dbt projects from your web browser. You can either use dbt Cloud's Developer package for free or pay for additional features. Learn more in dbt's Pricing documentation.
dbt CLI is a command line interface for running your dbt models. It is open source and free to use.
How does Fivetran work with dbt?
Fivetran connects to your Git provider and runs your dbt transformations according to the schedule you created in your dbt project. In your Fivetran dashboard, you can view the current status and historical logs of each dbt job. We sync your dbt transformations from your Git provider every few minutes to ensure that we are up to date.
Fivetran supports dbt transformations for the following destinations:
Fivetran dbt packages
Note: To use Fivetran's dbt packages, you must have a BigQuery, Redshift, or Snowflake destination.
We offer dbt packages for our top connectors. Our pre-built packages produce clear, comprehensible tables that power your reports, visualizations, and dashboards.
Fivetran has dbt packages for the following connectors:
- Asana (source / model)
- Fivetran Log Connector
- GitHub (source / model)
- Google Ads (source / model)
- HubSpot (source / model)
- LinkedIn Ad Analytics (source / model)
- Marketo (source / model)
- Microsoft Advertising (source / model)
- Netsuite SuiteAnalytics
- Pinterest Ads (source / model)
- Shopify (source / model)
- Stripe (source / model)
- Twitter Ads (source / model)
- Zendesk Support (source / model)
Note: Our older dbt packages have all necessary files combined into a single folder. Our newer dbt packages are split into two packages - source and model.
To add a Fivetran dbt package to your dbt project, see our dbt setup guide.
If you'd like Fivetran to create a dbt package for a connector that's not listed here, send your request to firstname.lastname@example.org.
How do Fivetran dbt packages work?
After Fivetran syncs your data to your destination, our dbt packages transforms your data and create analytics-ready tables in your destination.
Two dbt packages correspond to each supported connector:
- The source packages expose and document the underlying Fivetran schemas being created in the destination.
- The model packages reproduce commonly-recreated models for each source.
Follow our step-by-step setup guide to learn how to use dbt transformations in Fivetran.