MongoDB Replica Set Setup Guide

Follow these instructions to connect your MongoDB replica set to your destination using Fivetran.

Prerequisites

To connect your MongoDB database to Fivetran, you need:

MongoDB version 2.6 to 4.2
IP (e.g. 1.2.3.4) or host (your.server.com)
Port (usually 27017)

Note: Fivetran does not support Mongo tiers M0, M2, or M5 because MongoDB's smaller managed tiers do not provide oplogs. We need oplogs to perform incremental updates.

Allow access

This step allows Fivetran to access your database.

Connect to replica set node

Connect Fivetran to a node in your replica set. Replica set replication must be enabled on your MongoDB server. If it is not enabled, follow the replica set deployment instructions in the MongoDB manual.

Allow port access

Configure your firewall and/or other access control systems to allow incoming connections from Fivetran's IP or your SSH tunnel server's IP to your MongoDB port (default is 27017) .

SSL support

Fivetran's MongoDB connector supports encrypted connection to MongoDB over SSL if your database is set up to allow clients to connect without a certificate.

Create Fivetran user

Once Fivetran has access to your database, you must create a Fivetran user. Connect to the replica set or the primary node using the MongoDB shell as an admin user for Fivetran.

Fivetran's MongoDB connector expects to find this user in the admin database. Switch to the admin database and create the user there. Replace <password> with a password of your choice.

    use admin
    db.createUser({
      user: "fivetran",
      pwd: "<password>",
      roles: [ "readAnyDatabase" ]
    })

The Mongo shell command to create a user is db.AddUser() for versions 2.x.

For v3.4 and greater, the built-in role readAnyDatabase no longer includes the local database. (See Mongo's documentation for more details.) Instead, you must create the user with the following roles. Replace <password> with a password of your choice.

   use admin
   db.createUser({
     user: "fivetran",
     pwd: "<password>",
     roles: [ "readAnyDatabase", {role: "read", db: "local"} ]
   })

Choose schema prefix

Fivetran maps the schemas we discover in your source database to your destination and prepends the destination schema names with a prefix of your choice. For example, if your source database contains schemas foo and bar and if you choose the prefix pre, then your destination schemas will be pre_foo and pre_bar.

Fill in the hosts and ports field

In your connector setup form, fill the hosts and ports field in one of the following ways:

Enter the host name and port number for the primary node and all replicas. Use the plus icon to add new fields until all nodes have been added.

Each of these entries will be a host name or IP address that is appended by a colon and the port number (for example, server.example.com:27017 or 1.2.3.4:27017).
Enter the SRV host record for the MongoDB instance. This will be a mongodb+srv:// URL value, such as mongodb+srv://server.example.com/.

Note: Do not include any path contents after the top-level domain in this URL.

Connecting with MongoDB Atlas

In order to connect with MongoDB Atlas, follow the steps below:

Create an oplog user

Log in to the MongoDB Atlas dashboard.
In the Security section of the left menu, select Database Access.
Click New Database User.
Choose the password authentication method.
Enter a username and password for the new user.
In the Database User Privileges drop-down menu, select Grant Specific User Privileges.
Under Specific Privileges, add the following roles/privileges:
- readAnyDatabase
- read on the local database
Click Add User.

For general instructions, see MongoDB Atlas's documentation.

Connecting with mLab

In order to connect with mLab, follow the steps below:

Create an oplog user

Log in to the mLab management portal.
From your account’s Home page, navigate to the deployment whose oplog you want to access.
In the System Databases section, click the admin database.
Click the Users tab.
Click Add oplog user to create a new user.

You can find these instructions in mLab's documentation.

Note

Set the oplog size so that it can retain at least 24 hours' worth of changes. We recommend increasing the size to accommodate seven days' worth of data. See this MongoDB tutorial to learn how to adjust the size of your oplog or this MongoDB Atlas Doc

If you are using MongoDB version 4.0 or later, you must enable change streams on all collections. See this MongoDB tutorial to learn how.

mLab Dedicated plans are initially configured using MongoDB’s default oplog size, but you can customize the size. The oplog on dedicated plan deployments counts toward the pre-configured amount of storage that comes with each plan. For example, if your dedicated plan includes 60 GB of storage and you configure the oplog to be 10 GB, there will only be 50 GB left for use.

Connector Overview

Schema Information

Release Notes

API Connector Configuration

Documentation Home