How to Sync Data from Unsupported Sources
Use case
You want to sync data from a source that Fivetran does not have a connector for, typically a third-party application or vendor. You checked Fivetran’s connector directory and do not see your source listed as a current connector option or see it listed as Coming Soon.
TIP: You can upvote existing feature requests or submit new feature requests for your source. You can also check with your Fivetran account manager if a feature request already exists. If you do not have a Fivetran account manager, contact Fivetran support for your request.
Fivetran recommendation
Fivetran supports six workarounds for connecting to unsupported sources and loading data into a destination:
Do regular file exports from your source to cloud storage. Find a list of supported services in our Files documentation.
If your source application is built on a Fivetran-supported database that you can access, connect to that underlying database. Find a list of supported databases in our Databases documentation.
Use the Fivetran Connector SDK to code a custom data connector using Python. Learn more in our Connector SDK documentation.
Use custom cloud functions to pull data from the source API. Find a list of supported cloud functions in our Functions documentation.
Configure webhooks to push data. Learn more in our webhooks documentation.
Use our By Request program to build an unsupported SaaS application connector and centralize essential data from your business-critical SaaS applications. Learn more in our By Request documentation.
Search online to see which options work for your source. Some common web searches include the following:
- “[source name] + data export”
- “[source name] + API”
- “[source name] + webhooks”
If you’re not sure which of these options are available for your source, we recommend contacting your source vendor directly, sharing the options above, and asking them which one(s) they support.
Considerations
This section describes the pros and cons of each workaround. Depending on your source application, one workaround might be more appropriate than another.
File exports
File exports can generate full file refreshes or delta files for each new file created and delivered to Fivetran. You must pay careful attention to avoid unnecessarily high MAR when setting up file connectors in Fivetran. You can also use our Email connector to receive automated file reports generated from the source.
Database
Connecting to an underlying database has some added benefits because not only can you securely connect to the database, but you can also track the changes that occur in the source.
Connector SDK Beta
The Fivetran Connector SDK provides a custom-coded solution for fetching data from sources unsupported by our native connectors. Use it to code custom connectors tailored to your specific needs with complete control over the integration. The custom code runs on our infrastructure, so you don't need to use any additional external services with this solution.
If you want a custom connector built using Connector SDK but don't have technical development resources, reach out to Fivetran to discuss working with our Professional Services team or one of our consulting partners.
For more details, see our Connector SDK documentation.
Custom cloud functions
If you want custom cloud functions but don't have technical development resources, reach out to Fivetran to discuss the option of working with our Professional Services team or one of our consulting partners.
Webhooks
If an API that we support uses a webhook, we recommend using the API to connect because webhooks circumvents the transformation step of the ETL process. We have not optimized our webhooks connector for a specific source because we built it to be flexible to connect with as many APIs as possible.
We only store the events that we receive after you configure the webhook and cannot fetch historical event data from the web app.
If we don't have a connector for your source but that source has a webhooks API, you can use the source's webhooks to send event data to the Fivetran URL endpoint and we will sync the data to your destination.
By Request
If you want a SaaS application connector that Fivetran currently doesn't support or a connector as a service that uses HTTP APIs, we recommend that you submit a request to build a connector for the specific source.