Building custom pipelines doesn’t have to be hard. If you’re pulling from obscure sources or using custom APIs, our cloud function connectors help limit your engineering burden.
HOW THEY WORK
If you’re a developer, these steps should be easy. If not, we can put you in touch with one of our partners.
Using Go, Java, Node.js, Python, C# or F#, write a small function to fetch data from your custom source. Then write into our state logic to handle the incremental updating.
Host your code on Google Cloud Functions, AWS Lambda or Azure Functions.
After you fetch the data, we’ll load it into your warehouse, calling your function as often as every five minutes to fetch new data, deduplicate it and incrementally update it.
Common use cases for our cloud functions include fetching data from the following places:
Fivetran cloud function connectors improve on DIY with:
DATA SOURCE CONNECTOR DIRECTORY
Don’t see a data source you need? We add new connectors based on customer requests.
Stay up to date with new connectors and features as they become available.