DocumentDB Setup Guide

Follow our setup guide to connect Amazon DocumentDB to Fivetran.

Prerequisites

To connect your DocumentDB to Fivetran, you need:

• DocumentDB version 4.0 or higher
• Your database primary instance's IP (e.g., 1.2.3.4) or domain (your.server.com)
• Your database's port (usually 27017)
• An SSH server

NOTE: Fivetran needs to connect to the database's primary instance to perform incremental updates using change streams. (DocumentDB does not support reading change streams using replica instances).

To perform the initial syncs though, Fivetran can connect to replica instance(s) and recommends the same as it helps enhance the sync performance.

Setup instructions

Connect to DocumentDB

Create user

Create a database user for Fivetran using the DocumentDB Mongo shell.

1. Open the Mongo shell.

2. Connect to your primary node as an admin user.

3. Go to the admin database.

4. Execute the following command to create a user for Fivetran. Replace <username> and <password> with a username and password of your choice.

      use admin
      db.createUser({
        user: "<username>",
        pwd: "<password>",
        roles: [ "readAnyDatabase" ]
      })
   

Enable change streams

Fivetran uses change streams to perform incremental updates. You must explicitly enable change streams for all of the collections that you want Fivetran to sync.

Follow Amazon DocumentDB's Enabling Change Streams instructions to enable change streams.

Set change stream log retention duration

Set the change stream log retention duration so that it can retain at least 48 hours' worth of changes. We recommend increasing the size to retain seven days' worth of data.

Follow Amazon DocumentDB's Modifying the Change Stream Log Retention Duration instructions to adjust your change stream log retention duration.

Finish Fivetran configuration

1. In your connection setup form, enter a destination schema prefix. This prefix applies to each replicated schema and cannot be changed once your connection is created.

2. In the Host and ports field, enter the hostname and port number for your primary node. This should be a hostname or IP address that is appended by a colon and the port number (for example, server.example.com:27017 or 1.2.3.4:27017).

   • (Optional) Enter the hostname and port number for one or more of your replica nodes. Fivetran will use the replicas to improve the initial sync speed and balance resource utilization across the replicas.

NOTE: Do not include any path contents after the top-level domain in this URL.

3. Enter the Fivetran-specific user that you created in Step 2.

4. Enter the password for the Fivetran-specific user that you created in Step 2.

5. Choose your connection method. If you selected Connect via an SSH tunnel, copy or make a note of the Public Key and add it to the authorized_keys file while configuring the SSH tunnel, and provide the following information:

   • SSH hostname (do not use a load balancer's IP address/hostname)
   • SSH port
   • SSH user
   • If you enabled TLS on your database in Step 1, make sure to keep the Require TLS through Tunnel toggle turned ON.

6. Select your Pack mode.

7. Click Save & Test. Fivetran tests and validates our connection to your DocumentDB primary instance. Upon successful completion of the setup tests, you can sync your data using Fivetran.

Setup tests

Fivetran performs the following tests to ensure that we can connect to your DocumentDB primary and replica instances and that it is properly configured:

• The Connecting to SSH Tunnel Test validates the SSH tunnel details you provided in the setup form. It then checks that we can connect to your database using the SSH Tunnel.
• The Connecting to Host Test validates the database credentials you provided in the setup form. It then verifies that the database host is not private and checks that we can connect to the host.
• The Validating Certificate Test generates a pop-up window where you must choose which certificate you want Fivetran to use. It then validates that certificate and checks that we can connect to your database using TLS.
• The Connecting to Database Test connects to your all the database instances provided and verifies that your database is at least version 4.0. It then checks that we can access the schemas in your database.
• The Connecting to Database Test also checks if one of the instances provided is of type 'primary' by verifying that the change stream feature is enabled for at least one collection.

NOTE: The tests may take a few minutes to finish running.