Can Fivetran Troubleshoot the Issues in the Data Stored in My Destination?

Question

Can Fivetran troubleshoot the issues in the data stored in my S3 Data Lake destination?

Environment

S3 Data Lake destination

Answer

Yes, Fivetran can troubleshoot the issues in the data stored in your S3 Data Lake destination. However, to enable us to troubleshoot the issues, you must allow us to access your destination data.

To allow Fivetran to access the data in your destination, do the following:

1. Open your Amazon IAM console.

2. Go to Policies, and then click Create Policy.

   

3. Go to the JSON tab.

4. Depending on the format of your destination tables, copy one of the following policies and paste it in the JSON editor:

   • If your destination tables are in Iceberg format, copy the following policy:

     {
          "Version": "2012-10-17",
          "Statement": [
              {
              "Effect": "Allow",
              "Action": [
                  "athena:*"
              ],
              "Resource": [ "*" ]
              },
              {
              "Effect": "Allow",
              "Action": [
                  "s3:GetBucketLocation"
              ],
              "Resource": [
                  "arn:aws:s3:::{your-bucket-name}/{prefix-path}/*",
                  "arn:aws:s3:::{your-bucket-name}"
              ]
              },
              {
              "Effect": "Allow",
              "Action": [
                  "glue:GetDatabases"
              ],
              "Resource": [
                  "arn:aws:glue:{your-catalog-region}:{your-account-id}:catalog"
              ]
              }
          ]
      }
     

     NOTE: In the policy, replace the following:

     • {your-bucket-name} with the name of your S3 bucket
     • {prefix-path} with the prefix path of your S3 bucket
     • {your-catalog-region} with the Region of your S3 bucket
     • {your-account-id} with your AWS account ID

   • If your destination tables are in Delta Lake format, copy the following policy:

     {
          "Version": "2012-10-17",
          "Statement": [
              {
              "Effect": "Allow",
              "Action": [
                  "athena:*"
              ],
              "Resource": [ "*" ]
              },
              {
              "Effect": "Allow",
              "Action": [
                  "s3:GetBucketLocation"
              ],
              "Resource": [
                  "arn:aws:s3:::{your-bucket-name}/{prefix-path}/*",
                  "arn:aws:s3:::{your-bucket-name}"
              ]
              },
          ]
     }
     

     NOTE: In the policy, replace the following:

     • {your-bucket-name} with the name of your S3 bucket
     • {prefix-path} with the prefix path of your S3 bucket

5. Click Next: Tags.

6. Click Next: Review.

7. In the Name field, enter a name for your policy, and then click Create policy.

   NOTE: You can delete this policy after we complete the troubleshooting.

8. Go to Roles.

9. Open the IAM role you created for your destination.

   

10. In the Permissions tab, click Add Permissions, and then select Attach Policies.

    

11. Select the IAM policy you created for Athena.

12. Click Add Permissions.