Function connectors
fetch data from obscurity.
Building custom pipelines doesn’t have to be hard. If you’re pulling from obscure sources or using custom APIs, our cloud function connectors help limit your engineering burden.
Build a custom connector.
If you’re a developer, these steps should be easy. If not, we can put you in touch with one of our partners.
Step 1: Write a small function to fetch data
Using Go, Java, Node.js, Python, C# or F#, write a small function to fetch data from your custom source. Then write into our state logic to handle the incremental updating.
Step 2: Host your function on a serverless platform
Host your code on Google Cloud Functions, AWS Lambda or Azure Functions.
Step 3: Connect Fivetran
After you fetch the data, we’ll load it into your warehouse, calling your function as often as every five minutes to fetch new data, deduplicate it and incrementally update it.
We support cloud functions for these serverless platforms:
When to use cloud functions
Common use cases for our cloud functions include fetching data from the following places:
-
APIs without a pre-built connector
-
Private APIs
-
Data formats that don’t self-describe
-
Sensitive data that needs filtering or anonymizing
Advantages over DIY solutions
Fivetran cloud function connectors improve on DIY with:
-
Built-in version control, testing and infrastructure management
-
Compatibility with virtual private cloud (VPC)
-
Fewer engineering resources required
-
Fivetran expertise delivering incremental changes
- Cloud Functions
- View All
Connectors for every data source.
Request Connector
