Apache Kafka Requirements
This section describes the requirements, access privileges, and other features of Fivetran HVR when using Apache Kafka for replication.
Supported Platforms
- Learn about the Apache Kafka versions compatible with HVR on our Platform Support Matrix page (6.1.0, 6.1.5, 6.2.0, and 6.2.5).
Supported Capabilities
- Discover what HVR offers for Apache Kafka on our Capabilities for Apache Kafka page (6.1.0, 6.1.5, 6.2.0, and 6.2.5).
Data Management
- Learn how HVR maps data types between source and target DBMSes or file systems on the Data Type Mapping page. For the list of supported Apache Kafka data types and their mapping, see Data Type Mapping for Apache Kafka.
Installation Dependencies
On Linux, to use either of the Kafka authentication methods (User Name and Password or Kerberos), HVR requires the library libsasl2.so.2 to be installed. This library is part of Cyrus SASL and can be installed as follows:
$ yum install cyrus-sasl # On Red Hat Linux, CentOS $ zypper install cyrus-sasl # On SUSE Linux
There are no installation dependency for using Kafka authentication methods on Windows.
Known Issues
Kafka Broker Version 0.9.0.0 and 0.9.0.1
When using Kafka broker version 0.9.0.0 or 0.9.0.1, a known bug (KAFKA-3547) may cause a timeout error in HVR.
To resolve this issue, define action Environment for the Kafka location as follows:
| Group | Table | Action | Parameter(s) |
|---|---|---|---|
| KAFKA | * | Environment | Name=HVR_KAFKA_BROKER_VERSION Value=0.9.0.1 |
If you are using Kafka broker version 0.9.0.0, set Value=0.9.0.0
Amazon MSK
As noted in the AWS documentation, Amazon MSK only supports SCRAM-SHA-512 authentication.
To override HVR's default authentication method (PLAIN or GSAPI), define action Environment for the Kafka location as follows:
| Group | Table | Action | Parameter(s) |
|---|---|---|---|
| KAFKA | * | Environment | Name=HVR_KAFKA_PROPERTY Value=sasl.mechanism=SCRAM-SHA-512 |