Google Cloud Functions Setup Guide

Follow our setup guide to connect Google Cloud Functions to Fivetran.

Prerequisites

To connect Google Cloud Functions to Fivetran, you need a Cloud Function implementation.

Setup instructions

Develop function code

1. Using an IDE or any other tool, develop the code that you want to run in the Google Cloud Functions.

2. In the response header, add Content-Type with the value application/json.

For more information about Google Cloud Function implementation, see our Sample Google Cloud Function.

Create cloud function

1. Open the Google Cloud Functions console and click the projects drop-down list.

   

2. Select the project in which you want to create the function.

3. Click Create Function.

   

4. In the Environment drop-down menu, select the cloud function's generation: 1st gen or 2nd gen.

5. Enter the Function name.

6. In Region drop-down menu, select the region.

7. Based on your Environment, do the following in the Trigger section:

   • 1st gen

     1. In the Trigger type drop-down menu, select HTTP.
     2. In the Authentication section, select Require authentication.
     3. Click Save.

     

     NOTE: For more information about HTTP functions, see Google Cloud Functions documentation.

   • 2nd gen

     In the Authentication section, select Require authentication. Click Save.

     

8. In the RUNTIME tab, do the following:

   • In the Memory allocated drop-down menu, select the memory allocation size for the function.
   • Enter the Timeout.
   • Enter the Maximum number of instances.

   

9. In the CONNECTIONS tab, in the Ingress settings section, select Allow all traffic.

   

10. Click Next.

11. In the Runtime drop-down menu, select your runtime environment.

12. Select the Source code type.

    NOTE: We have used the Inline editor option to illustrate.

13. Specify the Entry point. This is the entry point for your function code execution.

    

14. Click Deploy.

Test cloud function

1. Open the cloud function you created in Step 2.

   

2. In the TESTING tab, add one example request and then click Test the function.

   

   NOTE: You can view the execution result in the Output field.

Get trigger name

1. Go to the TRIGGER tab.

   

2. Make a note of the Trigger URL. You will need it to configure Google Cloud Functions to connect with Fivetran.

Begin Fivetran configuration

1. In your connector setup form, enter the Destination schema name of your choice.

2. Enter the Function Trigger you found in Step 4.

3. (Optional) Click + Add secrets to specify your secrets as key-value pairs.

   • Enter your Secret Name. The secret name must be unique.
   • Enter your Secret Key associated with the secret name.

   For example, if you want to pass the secrets as {'apiKey': 'yourApiKey', 'consumerKey': 'test'}, add a key-value pair for each entry in the JSON structure. Make sure that the key names are unique in the key-value pairs.

   TIP: For Google Cloud Functions connectors created before August 9, 2023, enter your Secrets using the following JSON format:

   { "consumerKey": "", "consumerSecret": "", "apiKey": "yourApiKey" }

   NOTE: For more information on secrets, see our Function request and response documentation.

4. (Optional) Click + Add payload(s) to specify your custom payload(s) as key-value pair(s). Make sure that the key names are unique in the key-value pairs.

   • Enter your Payload Name. The payload name must be unique.
   • Enter your Payload Value associated with the payload name.

5. Make a note of the service account email address. You will need it to grant invoker permissions to your function.

Grant function permission

If in Step 2, you selected 1st gen as the Environment, do the following:

1. In the PERMISSIONS tab, click GRANT ACCESS.

   

2. In New principals field, add the service account email ID you found in Step 5.

3. In the Role drop-down menu, select Cloud Functions Invoker.

   

4. Click Save.

If in Step 2, you selected 2nd gen as the Environment, do the following:

1. In your Google Console, open Cloud Run.

2. Click SHOW INFO PANEL > ADD PRINCIPAL.

   

3. In New principals field, add the service account email ID you found in Step 5.

4. In the Role drop-down menu, select Cloud Run Invoker.

   

5. Click Save.

Finish Fivetran configuration

In your connector setup form, click Save & Test.

Fivetran tests and validates the Google Clould Functions connection. On successful completion of the setup tests, you can sync the data returned by the Google Cloud function to your destination.

Setup tests

Fivetran performs the following Google Cloud Functions connection tests:

• The Validate Secrets test checks if you have entered the secrets in a valid JSON format.
• The Validate Payloads test checks if you have entered the payloads in a valid JSON format.
• The Function Connection test checks if we can connect to your Google Cloud Function and if the function’s response format is correct.

IMPORTANT: The setup tests may timeout if your function’s response size is more than 500 megabytes (MB). In the setup tests request, Fivetran passes the setup_test field as true; and your function must return a lightweight JSON object with the hasMore field as false, not exceeding 500 MB in size. The 500 MB size limitation is only applicable to the setup tests and not for your syncs.

Related articles

description Connector Overview

assignment Release Notes

settings API Connector Configuration

home Documentation Home