DynamoDB Setup Guide link

Updated November 16, 2023

Follow our setup guide to connect DynamoDB to Fivetran.

Prerequisiteslink

To connect your DynamoDB database to Fivetran, you need an AWS account with administrator privileges.

Setup instructionslink

Find External IDlink

Find the automatically-generated External ID in your connector setup form and make a note of it. You will need it to configure AWS to connect with Fivetran.

NOTE: The automatically-generated External ID is tied to your account. If you close and re-open the setup form, the ID will remain the same. You can keep the tab open in the background while you configure your source for convenience.

Create IAM policylink

This step allows Fivetran to access your DynamoDB database.

Open the Create new AWS IAM policy page.
Go to the JSON tab.

Copy the following policy and paste it in the JSON editor:

{
    "Version": "2012-10-17",
    "Statement": [
        {
        "Effect": "Allow",
        "Action": [
                "dynamodb:DescribeStream",
                "dynamodb:DescribeTable",
                "dynamodb:GetRecords",
                "dynamodb:GetShardIterator",
                "dynamodb:ListTables",
                "dynamodb:Scan"
        ],
        "Resource": "*"
        }
    ]
}
content_copy

NOTE: This policy provides us access to all the source tables. However, you can modify the policy to restrict access to only specific tables.

IMPORTANT: If you use a customer-managed KMS key, add the following actions to the Action section of the IAM policy to provide read access to the encrypted tables:

 "kms:Decrypt"
 "kms:Encrypt"
 "kms:GenerateDataKey"
 "kms:ReEncryptTo"
 "kms:GenerateDataKeyWithoutPlaintext"
 "kms:DescribeKey"
 "kms:ReEncryptFrom"
content_copy

Click Next: Tags.
(Optional) In the Add tags page, add custom tags to your DynamoDB database.
Click Next: Review.
In the Review policy page, enter a name for the policy (for example, Fivetran-Dynamo-Access).
(Optional) Provide a description for the policy.
Click Create policy.

Create IAM rolelink

Open the Create new AWS IAM role page.
Select AWS account and enter Fivetran’s AWS VPC Account ID, 834469178297, in the Account ID field.
Select the Require external ID checkbox and enter the External ID you found in Step 1, then click Next.
In the Add permissions page, select the IAM policy you created in Step 2, and then click Next.
Enter a name for the role (for example, Fivetran-Dynamo), and then click Create role.
Click the role you just created (it may take a few seconds to populate). Find the "Role ARN" and copy it. Enter this value in the Role ARN field of your connector setup form.

Enable Streams for DynamoDB tableslink

In this step, you'll enable streams for all the tables that you want to sync through Fivetran.

In your AWS console, select the DynamoDB service, and then select Tables.
Select a table.
Go to the Exports and streams tab.
In the DynamoDB stream details section, click Turn on.
Select New and old images - both the new and the old images of the item.
Click Turn on stream.
Repeat steps 1 through 6 for every table that you want to sync using our connector.

(Optional) Configure AWS PrivateLink Betalink

IMPORTANT: You must have a Business Critical plan to use AWS PrivateLink.

AWS PrivateLink allows VPCs and AWS-hosted or on-premises services to communicate with one another without exposing traffic to the public internet. PrivateLink is the most secure connection method. Learn more in AWS’ PrivateLink documentation.

Follow our AWS PrivateLink setup guide to configure PrivateLink for your database.

Finish Fivetran configurationlink

Enter your chosen destination schema name in the connector setup form.
Select your AWS region.
Select your pack mode.
(Optional) To always connect using AWS PrivateLink, set the Require PrivateLink toggle to ON.
NOTE: By default, we use PrivateLink to connect if your database and Fivetran are in the same region. Enabling this option ensures that we always use PrivateLink to connect. If the regions are different, Fivetran won't create the connection. When you enable the option, the requests to your DynamoDB endpoint within the AWS Region are routed to a private DynamoDB endpoint within the Amazon network. You don't need to modify your applications running on EC2 instances in your VPC. The endpoint name remains the same, but the route to DynamoDB stays entirely within the Amazon network and does not access the public internet.
Click Save & Test. Fivetran will take it from here and sync your data from your DynamoDB account.

Setup testslink

Fivetran performs the following tests to ensure that we can connect to your generic DynamoDB database:

The Connecting to Database Test checks that we can access your DynamoDB database using the credentials you provided in the setup form.
The Source and Destination Region Uniformity Test validates if your database and our vpc endpoint are in the same region. We skip this test if you haven't enabled the Require PrivateLink toggle. The test fails if your source and fivetran's vpc endpoint are in different regions.

NOTE: The tests may take a few minutes to finish running.

Documentation