How Can I View Detailed Logs for My Cloud Function?

Use Case

You are experiencing issues with your cloud function but the logs provided by Fivetran doesn't indicate where the root cause lies.

Recommendation

Make sure that you are printing out errors throughout your function, specifically at points where you receive data in response or transform that data in some way (such as parsing or deserializing JSON). Use custom error handling to send error responses to Fivetran.

Context

When Fivetran interacts with a cloud function, the cloud platform makes a request to invoke the function (in the form of a request object) and waits for a response back (in the form of a response object).

You may experience one of the following three groups of errors:

• 4xx errors
• 5xx errors
• Non-4xx or 5xx errors

Cause

The errors indicate different root causes:

• 4xx client errors indicate the following:

  • A permissions error
  • An issue with the configured JSON in the secrets object

• 5xx server errors indicate the following:

  • The server failed to fulfill a request
  • Gateway issue or timeout

• Other errors (non-4xx or 5xx errors). For example:

  • Unable to parse JSON response due to incorrect response format
  • There's no response for the request sent by Fivetran (NoHttpResponseException)

Rectification steps

4xx errors

To fix the errors, do one of the following:

• Fix the permissions in the cloud function platform
• Fix the function's configuration in the connector setup form

5xx errors

For 5xx errors, we add an Error alert 'Fix Cloud Function' on the connector dashboard. The notification contains the payload that caused the connector to break.

However, note that the rectification steps for 5xx errors are not straightforward; Fivetran has successfully invoked the function, but it hasn’t been completed for an unknown reason, and the error occurred during the function execution.

If you have added logging throughout your function, you can do the following:

1. Run your function within the cloud environment.
2. Use your logging to identify and fix the line(s) of code.
3. Re-run the cloud function connector.

Non-4xx or 5xx errors

Contact our support team.

Considerations

If your cloud function incurs a 5xx error, the data sync stops until you fix the function.

You can fix and re-deploy the function within your cloud platform. You don't need to pause your connector. If you have built idempotency into your function, Fivetran resumes the sync from where it stopped because of the connector error.