When Fivetran Doesn't Support Your Source

It's possible to load data from unsupported sources into your warehouse, and we're always adding connectors based on customer demand.

By Jeff Tietz, 3 Jul, 2019

If Fivetran hasn’t built a connector for your data source, you can use one of the four data loading methods outlined below — but we’re continually building new connectors, so be sure to ask your Fivetran account manager whether the one you need is already under construction. If not, please let us know which connectors you need by reaching out to your account manager, contacting sales@fivetran.com or logging in to your account and filling out our connector request form:

If your data source isn’t on our roadmap yet, here are your options for loading data into a destination:

  • File replication. Convert your data into files that can connect to your destination using one of the Fivetran file replication services.

  • Webhooks. Using the Fivetran webhooks connector, send data to your destination from applications that support webhooks.

  • Database applications. If your application sits on top of an underlying database, send data to Fivetran from applications using one of the many Fivetran database connectors.

  • Cloud functions. If you have access to your source’s API, you can write a custom cloud function allowing Fivetran to read the data from the application and write it to your destination. Email. If your application supports sending a report in the form of a csv (zipped or unzipped) attachment, Fivetran can read this data and write it to your specified table in the destination.

File Replication

Fivetran can ingest data through the following file storage and transfer services:

  • Amazon Cloudfront

  • Amazon Kinesis

  • Amazon S3

  • Azure Blob Storage

  • Fivetran native browser uploader

  • Dropbox

  • Fivetran native email ingestor

  • FTP, FTPS, SFTP

  • Google Cloud Storage

  • Google Sheets

Just drop the files into any of these services and we'll bring the data into your destination.

Many Fivetran users are able to automate loading data into a file storage service, which is often easier and more robust than trying to automate loading directly into a destination. Fivetran file connectors load data into your destination and continuously update and normalize it.

To learn more about specific file replication connectors, visit our documentation.

Webhooks

A webhook is an HTTP callback triggered by an event on your website or application. While webhooks cannot pull historical data, they’re a great way to gather new data. Most SaaS applications support webhooks, but you will need to confirm that capability with the specific application.

To get set up, Fivetran will give you a URL that you put inside your SaaS service. As new events occur in your applications, the SaaS service will post data to that URL, which Fivetran will collect and ingest into your destination. Please note that webhooks are developer tools; to implement them correctly, users need a minimum level of expertise.

To learn more, visit our webhooks documentation.

Database Applications

Fivetran supports most popular databases as data connectors. Many applications are built on top of these databases. If Fivetran supports the underlying database, we can connect to it and collect data from your application. To find out if your application can be supported in this way, reach out to your application contact or do a simple internet search to identify the underlying database of your source. When researching the underlying database, be sure to check pre-configuration documents to confirm permission requirements for connecting to your software program through the database.

Fivetran supports connectors for the following databases:

  • PostgreSQL

  • Azure

  • Oracle

  • MongoDB

  • SQL Server

  • MySQL

  • MariaDB

To learn more about database connectors, visit our documentation page.

Cloud Functions

A functions connector allows you to code a custom data connector as an extension of Fivetran. Fivetran will execute custom code in your account on a cloud platform and sync the returned data to your destination. For example, if you have a custom-built application with a private REST API, and want to use Fivetan to perform analytics on the data, you can write a cloud function to read data from the application and use the Fivetran connector to write data to your destination. Fivetran supports cloud functions from AWS Lambda, Azure Functions and Google Cloud Functions.

To learn more about cloud functions connectors, visit our documentation page.

Email

An email connector generates an email address that you can send supported file types to for Fivetran to ingest. We’ll determine the table structure according to the column types that are read from the file, and load into the table specified during your connector setup. To learn more, visit our email connector documentation.

Additional Resources

If you don’t want to utilize internal resources, Fivetran has a global network of partners with the expertise to help you deploy any of these methods. Just get in touch with your account manager and ask about the right consultant for your use case and needs, or reach out to sales@fivetran.com.

Start analyzing your data in minutes, not months

Launch any Fivetran connector instantly.

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

Adblock Detection