Confluent Cloud Destination Setup Guidelink
Updated February 5, 2024
Follow our setup guide to connect Confluent Cloud to Fivetran.
To connect Confluent Cloud to Fivetran, you need the following:
- A Confluent Cloud account with an active Kafka server
- Permissions to generate access keys for the broker and schema registry
Find Stream Governance API Endpointlink
Log in to Confluent Cloud.
Click View environments.
Select the environment that contains your Kafka cluster.
Make a note of the Stream Governance API Endpoint. You will need it to configure Fivetran.
Find Bootstrap server nameslink
In the Clusters tab, select your cluster.
On the navigation menu, click Cluster Settings.
Make a note of the Bootstrap server names that appear in the Endpoints section. You will need them to configure Fivetran.
Complete Fivetran configuration link
Log in to your Fivetran account.
Go to the Manage Account page.
In the Destinations tab, click Add Destination.
On the Add Destination To Fivetran page, enter a Destination name of your choice.
Select Confluent Cloud as the destination type.
In the destination setup form, enter the names of the Bootstrap Servers you found in Step 2. The server names must be in
NOTE: Click + to add more than one server name.
Enter your SASL API Key and API Secret.
Enter the number of Partitions you'd like to create in your topic. Partitions allow us to split the data of a topic across multiple brokers and balance the load between them.
NOTE: A consumer group can consume data from only one partition. Therefore, the number of consumer groups that consume data in parallel depend on the number of partitions in the topic. For more information about the partition limits applicable to your cluster type, see Confluent Cloud's documentation.
Enter a Replication Factor to specify the number of replicas you'd like to create for each of your topic partitions.
NOTE: The partition replicas increase the reliability and fault tolerance of the connection. The minimum and maximum supported values are
4, respectively. However, for best performance, we recommend that you set the Replication Factor to
Select your Data Format: JSON or AVRO.
NOTE: You cannot change the data format after you set up the connection.
In the Schema Registry URL field, enter the Stream Governance API Endpoint you found in Step 1.
Enter the Schema API Registry Key and Schema Registry API Secret.
Choose your Data processing location.
Choose your Cloud service provider and its region as described in our Destinations documentation.
Choose your Time zone.
(Optional for Business Critical accounts) To enable regional failover, set the Use Failover toggle to ON, and then select your Failover Location and Failover Region. Make note of the IP addresses of the secondary region and safelist these addresses in your firewall.
Click Save & Test.
Fivetran tests and validates the Confluent Cloud connection. On successful completion of the setup test, you can sync your data using Fivetran connectors to the Confluent Cloud destination.
Fivetran performs the Connecting to Kafka test to validate the broker credentials and to check if we can access your topic schema through the schema registry.
NOTE: The test may take a couple of minutes to complete.