The 2nd Annual Modern Data Stack Conference is back on September 22 - 23, 2021. Learn more and register here.
Function connectors fetch data from obscurity.
Building custom pipelines doesn’t have to be hard. If you’re pulling from obscure sources or using custom APIs, our cloud function connectors help limit your engineering burden.
HOW THEY WORK
Build a custom connector.
If you’re a developer, these steps should be easy. If not, we can put you in touch with one of our partners.
- See sample functions and more information
- Step 1: Write a small function to fetch data
Using Go, Java, Node.js, Python, C# or F#, write a small function to fetch data from your custom source. Then write into our state logic to handle the incremental updating.
- Step 2: Host your function on a serverless platform
Host your code on Google Cloud Functions, AWS Lambda or Azure Functions.
- Step 3: Connect Fivetran
After you fetch the data, we’ll load it into your warehouse, calling your function as often as every five minutes to fetch new data, deduplicate it and incrementally update it.
We support cloud functions for these serverless platforms:
When to use cloud functions
Common use cases for our cloud functions include fetching data from the following places:
- APIs without a pre-built connector
- Private APIs
- Data formats that don’t self-describe
- Sensitive data that needs filtering or anonymizing
Advantages over DIY solutions
Fivetran cloud function connectors improve on DIY with:
- Built-in version control, testing and infrastructure management
- Compatibility with virtual private cloud (VPC)
- Fewer engineering resources required
- Fivetran expertise delivering incremental changes
DATA SOURCE CONNECTOR DIRECTORY
Connectors for every data source.
Request Connector
Don’t see a data source you need? We add new connectors based on customer requests.
View Articles from our Blog
Stay up to date with new connectors and features as they become available.
Start analyzing your data in minutes, not months
Launch any Fivetran connector instantly.