MongoDB Sharded Cluster Setup Guide

Follow these instructions to connect your MongoDB sharded cluster to your destination using Fivetran.

Prerequisites

To connect MongoDB sharded cluster to Fivetran, you need the following:

• Permissions to add a database user:

  • MongoDB Atlas: You must have either Organization Owner or Project Owner role.
  • MongoDB shell: You must have a dbOwner, userAdmin or userAdminAnyDatabase role.

• To be able to identify your MongoDB sharded cluster's host identifier. The sharded cluster's host identifier has one of the following formats:

  Format	Example
  SRV host identifier	mongodb+srv://server.example.com
  Connection string	mongodb://query-router-00.example.com:27016,query-router-01.example.com:27016,query-router-02.example.com:27016
  Domain and port of the mongos query router	query-router-00.example.com:27016
  IP address and port of the mongos query router	1.2.3.4:27016

  NOTE:

  • For SRV host identifier and Connection string, we support connection tags

  • Optionally, you can use Analytics nodes to isolate queries from your operational workload and improve cluster reliability and performance, by specifying a connection string similar to the following example: mongodb+srv://example.mongodb.net/?readPreference=secondary&readPreferenceTags=nodeType:ANALYTICS

  • Optionally, you can specify the read preference in the connection string based on your priority as shown below:

    Read Preference	Priority
    secondaryPreferred(default)	Optimal performance
    primaryPreferred or secondaryPreferred	High availability
    nearest	Optimal performance for geographically distributed data

  • Using the primary/primaryPreferred read preference might affect operational performance of your cluster

Setup instructions

IMPORTANT: Do not perform the Choose connection method step if you want to use Hybrid Deployment for your data pipeline.

Find host identifiers

Find your host identifiers using either MongoDB Atlas or the MongoDB shell.

Allow database access

Create a database user for Fivetran using either MongoDB Atlas or the MongoDB shell.

use admin
db.createUser({
   user: "<username>",
   pwd: "<password>",
   roles: [ "readAnyDatabase", "clusterMonitor" ]
})

Choose connection method

Decide whether to connect Fivetran to your MongoDB cluster directly, using an SSH tunnel, or using a private link. For more information about the connection options, see How to Choose the Right Database Connection Option.

TIP: To enable TLS on your replica set, follow MongoDB's TLS Configuration instructions.

(Optional) Set oplog size

Set the oplog size so that it can retain at least 24 hours' worth of changes. We recommend increasing the size to retain seven days' worth of data.

Adjust your oplog size using either MongoDB Atlas or the MongoDB shell:

• MongoDB Atlas: Follow MongoDB Atlas' Set Oplog Size tutorial.
• MongoDB shell: Follow MongoDB's Change the Size of the Oplog tutorial.

Choose pack mode

Choose between packed mode and unpacked mode. Packed mode is the default setting.

In unpacked mode, we unpack one layer of nested fields and infer types. In packed mode, we write the data to your destination without unpacking.

TIP: Learn more about each pack mode and see example tables in our Pack mode documentation.

Finish Fivetran configuration

1. In your connector setup form, enter a Destination schema prefix. This prefix cannot be changed once your connector is created.

2. In the Host and port fields, enter the cluster's host and port.

3. Enter the username of the Fivetran-specific User that you created in the Allow database access step.

4. Enter the Password for the Fivetran-specific user that you created in the Allow database access step.

5. (Hybrid Deployment only) If your destination is configured for Hybrid Deployment, the Hybrid Deployment Agent associated with your destination is pre-selected in the Select an existing agent drop-down menu. To use a different agent, select the agent of your choice, and then select the same agent for your destination.

6. (Optional for Hybrid Deployment) If you want to use a TLS connection between the connector and your source database in your own environment, set the Require TLS toggle to ON.

   IMPORTANT: Before you set this toggle to ON, follow MongoDB's TLS Configuration instructions to enable TLS on your database. We do not support TLS 1.0.

7. (Not applicable to Hybrid Deployment) Choose your Connection Method. If you enabled SSL/TLS on your database in the Choose connection method step, set the Require TLS through Tunnel toggle to ON. If you selected Connect via an SSH tunnel, provide the following information:

   • SSH hostname (do not use a load balancer's IP address/hostname)
   • SSH port
   • SSH user

8. Select the pack mode you chose in the Choose pack mode step.

9. Select the Binary Data Representation corresponding to how you store binary data on your source server (either Base64 or UUID). This setting determines how we sync binary data types when we receive them from your source database.

   TIP: Learn more about binary data representation in our Type Transformations and Mapping documentation.

10. (Not applicable to Hybrid Deployment) Safelist the Fivetran IPs included in the setup form. You can choose either the IP Address or CIDR format.

11. Click Save & Test. Fivetran tests and validates our connection to your MongoDB sharded cluster. Upon successful completion of the setup tests, you can sync your data using Fivetran.

Setup tests

Fivetran performs the following tests to ensure that we can connect to your MongoDB cluster and that it is properly configured:

• The Connecting to SSH Tunnel Test validates the SSH tunnel details you provided in the setup form. It then checks that we can connect to your database using the SSH Tunnel. (We skip this test if you aren't connecting using SSH.)
• The Validate Host Test validates the database credentials you provided in the setup form. It then verifies that the database host is not private and checks that we can connect to the host.
• The Connecting to Host & Validating Certificate Test connects to your database instance and checks if we can access the schemas in your database.The Test generates a pop-up window where you must choose which certificate you want Fivetran to use. Select the root certificate to trust all the configured hosts. It then validates that certificate and checks that we can connect to your database using TLS. (If you selected an indirect connection method and then disabled the Require TLS through Tunnel toggle, then we do not generate the pop-up window and validated your certificate)
• The Database Access Test verifies if we have the permissions to query at least one collection.
• The Change Streams/Oplog Access Test checks if we can access the change streams. If we can't access the change streams, it then verifies if we can access the oplog and if the oplog contains at least 24 hours' worth of changes.
• The Validating Selection of Service Type test checks that you selected the correct service type for your Mongo configuration.

NOTE: The tests may take a few minutes to finish running.

Related articles

description Connector Overview

account_tree Schema Information

assignment Release Notes

settings API Connector Configuration

home Documentation Home