Documentation

Documentation

  • Getting Started
  • Core Concepts
  • Using Fivetran
  • Usage-Based Pricing
  • Connectors
  • Applications
  • Databases
    • Connection Options
    • Databases Troubleshooting
    • BigQuery
    • Convex
    • Cosmos DB
    • Db2 for i
    • DocumentDB
    • DynamoDB
      • Setup Guide
      • DynamoDB Troubleshooting
      • Release Notes
    • Elasticsearch
    • Firebase
    • MariaDB
    • MongoDB
    • MySQL
    • OpenSearch
    • Oracle
    • PlanetScale
    • PostgreSQL
    • Redshift
    • SAP ERP
    • Snowflake
    • SQL Server
    • Release Notes
  • Files
  • Events
  • Functions
  • Destinations
  • Partner-Built
  • Transformations
  • Logs
  • Security
  • REST API
  • Local Data Processing (HVR 6)
  • Release Notes
RSS
Release notes RSS
HVR 5 Documentation
HVR 5 Documentation
  • Support
  • Sign In
Edit on GitHub

DynamoDB Setup Guide link

Updated November 16, 2023

Follow our setup guide to connect DynamoDB to Fivetran.


Prerequisiteslink

To connect your DynamoDB database to Fivetran, you need an AWS account with administrator privileges.


Setup instructionslink

Find External IDlink

Find the automatically-generated External ID in your connector setup form and make a note of it. You will need it to configure AWS to connect with Fivetran.

NOTE: The automatically-generated External ID is tied to your account. If you close and re-open the setup form, the ID will remain the same. You can keep the tab open in the background while you configure your source for convenience.

Create IAM policylink

This step allows Fivetran to access your DynamoDB database.

  1. Open the Create new AWS IAM policy page.

  2. Go to the JSON tab.

    DynamoDB-click-json

  3. Copy the following policy and paste it in the JSON editor:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
            "Effect": "Allow",
            "Action": [
                    "dynamodb:DescribeStream",
                    "dynamodb:DescribeTable",
                    "dynamodb:GetRecords",
                    "dynamodb:GetShardIterator",
                    "dynamodb:ListTables",
                    "dynamodb:Scan"
            ],
            "Resource": "*"
            }
        ]
    }
    content_copy

    NOTE: This policy provides us access to all the source tables. However, you can modify the policy to restrict access to only specific tables.

    IMPORTANT: If you use a customer-managed KMS key, add the following actions to the Action section of the IAM policy to provide read access to the encrypted tables:

     "kms:Decrypt"
     "kms:Encrypt"
     "kms:GenerateDataKey"
     "kms:ReEncryptTo"
     "kms:GenerateDataKeyWithoutPlaintext"
     "kms:DescribeKey"
     "kms:ReEncryptFrom"
    content_copy
  4. Click Next: Tags.

  5. (Optional) In the Add tags page, add custom tags to your DynamoDB database.

  6. Click Next: Review.

  7. In the Review policy page, enter a name for the policy (for example, Fivetran-Dynamo-Access).

  8. (Optional) Provide a description for the policy.

  9. Click Create policy.

Create IAM rolelink

  1. Open the Create new AWS IAM role page.

  2. Select AWS account and enter Fivetran’s AWS VPC Account ID, 834469178297, in the Account ID field.

    DynamoDB-another-aws-account

  3. Select the Require external ID checkbox and enter the External ID you found in Step 1, then click Next.

    DynamoDB-next

  4. In the Add permissions page, select the IAM policy you created in Step 2, and then click Next.

    DynamoDB-select-fivetran-policy

  5. Enter a name for the role (for example, Fivetran-Dynamo), and then click Create role.

  6. Click the role you just created (it may take a few seconds to populate). Find the "Role ARN" and copy it. Enter this value in the Role ARN field of your connector setup form.

    DynamoDB-copy-arn-role

Enable Streams for DynamoDB tableslink

In this step, you'll enable streams for all the tables that you want to sync through Fivetran.

  1. In your AWS console, select the DynamoDB service, and then select Tables.

  2. Select a table.

  3. Go to the Exports and streams tab.

  4. In the DynamoDB stream details section, click Turn on.

  5. Select New and old images - both the new and the old images of the item.

  6. Click Turn on stream.

  7. Repeat steps 1 through 6 for every table that you want to sync using our connector.

    DynamoDB-enable-data-streams


(Optional) Configure AWS PrivateLink Betalink

IMPORTANT: You must have a Business Critical plan to use AWS PrivateLink.

AWS PrivateLink allows VPCs and AWS-hosted or on-premises services to communicate with one another without exposing traffic to the public internet. PrivateLink is the most secure connection method. Learn more in AWS’ PrivateLink documentation.

Follow our AWS PrivateLink setup guide to configure PrivateLink for your database.


Finish Fivetran configurationlink

  1. Enter your chosen destination schema name in the connector setup form.

  2. Select your AWS region.

  3. Select your pack mode.

  4. (Optional) To always connect using AWS PrivateLink, set the Require PrivateLink toggle to ON.

    NOTE: By default, we use PrivateLink to connect if your database and Fivetran are in the same region. Enabling this option ensures that we always use PrivateLink to connect. If the regions are different, Fivetran won't create the connection. When you enable the option, the requests to your DynamoDB endpoint within the AWS Region are routed to a private DynamoDB endpoint within the Amazon network. You don't need to modify your applications running on EC2 instances in your VPC. The endpoint name remains the same, but the route to DynamoDB stays entirely within the Amazon network and does not access the public internet.

  5. Click Save & Test. Fivetran will take it from here and sync your data from your DynamoDB account.

Setup testslink

Fivetran performs the following tests to ensure that we can connect to your generic DynamoDB database:

  • The Connecting to Database Test checks that we can access your DynamoDB database using the credentials you provided in the setup form.
  • The Source and Destination Region Uniformity Test validates if your database and our vpc endpoint are in the same region. We skip this test if you haven't enabled the Require PrivateLink toggle. The test fails if your source and fivetran's vpc endpoint are in different regions.

NOTE: The tests may take a few minutes to finish running.


Related articleslink

description Connector Overview

account_tree Schema Information

assignment Release Notes

settings API Connector Configuration

home Documentation Home


Questions?

We're always happy to help with any other questions you might have! Send us an email.

    Thanks for your feedback!
    Was this page helpful?