Location Connection for Apache Kafka

This section describes the connection details required for creating Kafka location in Fivetran HVR. For information about the pre-requisites, access privileges, and other configuration requirements for creating Kafka location, see Apache Kafka Requirements.

HVR uses librdkafka (C library which talks Kafka's protocol) to connect to Kafka.

If the managed secrets feature is enabled, option USE TOKEN INSTEAD is displayed in the fields designated for entering secrets.

Field	Description	Equivalent Location Property
BROKER(S) AND PORT(S)	Hostname or IP address of the Kafka broker server(s) along with the TCP port that the Kafka server uses to listen for client connections. The default port is 9092. HVR supports connecting to more than one Kafka broker servers. Click to add more Kafka brokers. When connecting to Kafka clusters in Confluent Cloud, use the Bootstrap server from the Cluster settings section of the Confluent Cloud web interface. Example: pkc_lgwgm.eastus2.azure.confluent.cloud When connecting to Azure Event Hubs, use the endpoint for your namespace. Example: demohub.servicebus.windows.net The port for Azure Event Hubs is always 9093.	Kafka_Brokers
AUTHENTICATION METHOD	Authentication method for connecting HVR to Kafka server (BROKER). Available options are: No Authentication User Name and Password Kerberos On Linux, to use either of the Kafka authentication methods (User Name and Password or Kerberos), HVR requires the library libsasl2.so.2 to be installed. For more information, see section Installation Dependencies in Apache Kafka Requirements. When connecting to Kafka clusters in Confluent Cloud, set the authentication mode to User Name and Password.	Kafka_Authentication_Method
USER	Username for connecting HVR to the Kafka server. This field is enabled only if AUTHENTICATION METHOD is set to User Name and Password. When connecting to Kafka clusters in Confluent Cloud, use the API access key obtained from the Confluent Cloud web interface. When connecting to Azure Event Hubs, use $ConnectionString.	Stream_User
PASSWORD	Password of the USER. When connecting to Kafka clusters in Confluent Cloud, use the API access secret obtained from the Confluent Cloud web interface. When connecting to Azure Event Hubs, use the following pattern for the password (this should be a one line without spaces): Endpoint=endpoint; SharedAccessKeyName=keyName; SharedAccessKey=accessKey where: endpoint is the namespace URL, which can be copied from the Azure Shared Access Policy tab on the left panel in the Azure portal. keyName is RootManageSharedAccessKey. accessKey is a security key associated with the Shared Access Policy and used to access your event hub. Example (this should be a one line without spaces): Endpoint=sb://demohub.servicebus.windows.net/; SharedAccessKeyName=RootManageSharedAccessKey; SharedAccessKey=xxxxxxxxxxxxx;	Stream_Password
SERVICE NAME	Kerberos Service Principal Name (SPN) of the Kafka server. This field is enabled only if AUTHENTICATION METHOD is set to Kerberos.	Kafka_Kerberos_Service
CLIENT PRINCIPAL	Full Kerberos principal of the client connecting to the Kafka server. This is required only on Linux/Unix. This field is enabled only if AUTHENTICATION METHOD is set to Kerberos.	Kafka_Kerberos_Client_Principal
CLIENT KEYTAB	Directory path where the Kerberos keytab file containing key for the CLIENT PRINCIPAL is located. This field is enabled only if AUTHENTICATION METHOD is set to Kerberos.	Kafka_Kerberos_Keytab
DEFAULT TOPIC	Kafka topic to which the messages are written. You can use strings/text or expressions as Kafka topic name. Following are the expressions to substitute capture location or table or schema name as topic name: {hvr_cap_loc} - for capture location name. {hvr_tbl_name} - for current table name. This is only allowed if the channel is defined with tables. {hvr_schema}- for schema name of the table. This is only allowed if the channel contains tables that have action TableProperties with parameter Schema=my_schema explicitly defined for these tables on the target file location. The Kafka topics should either exist already in the Kafka broker, or it should be configured to auto-create Kafka topics when HVR sends a message. When connecting to Azure Event Hubs, use your Event Hub name (e.g. myeventhub). This is optional.	Kafka_Default_Topic

Advanced Settings

Field	Description	Equivalent Location Property
Enable SSL	Enable/disable (one way) SSL. If enabled, HVR authenticates the Kafka server by validating the SSL certificate shared by the Kafka server. When connecting to Azure Event Hubs, SSL must be enabled if you have chosen User Name and Password in the AUTHENTICATION METHOD field.
CLIENT PUBLIC CERTIFICATE	Directory path where the .pem file containing the client's SSL public certificate is located.	Stream_Client_Public_Certificate
CLIENT PRIVATE KEY	Directory path where the .pem file containing the client's SSL private key is located.	Stream_Client_Private_Key
CLIENT PRIVATE KEY PASSWORD	Password of the private key file that is specified in CLIENT PRIVATE KEY.	Stream_Client_Private_Key_Password
TRUSTED CA CERTIFICATE	When connecting to Kafka clusters in Confluent Cloud or Azure Event Hubs, a Broker CA must be specified here. HVR supplies the CA in HVR_HOME/etc/cert/ca-bundle.crt.	Stream_Public_Certificate