Elastic Cloud Setup Guide Beta

Follow our setup guide to connect Elastic Cloud to Fivetran.

Prerequisites

To connect your Elastic Cloud account to Fivetran, you need:

An Administrator account
Elasticsearch version 7.10 or above

Setup instructions

Choose connection method

First, decide whether to connect Fivetran to your Elastic Cloud instance directly or by using an SSH tunnel.

Connect directly

Fivetran connects directly to your Elasticsearch cluster. This is the simplest method.

Connect using SSH

Fivetran connects to a separate server in your network that provides an SSH tunnel to your database.

If you connect using SSH, you will configure your tunnel server's security group to allow Fivetran access and configure your database's security to allow access from the tunnel.

Before you proceed to the next step, you must follow our SSH connection instructions.

Choose authentication method

Fivetran uses two authentication mechanisms to reach your Elastic Cloud instance, basic authentication and API Key.

Basic authentication

For a basic access authentication, you need to provide a user name and password to connect your Elastic Cloud instance to Fivetran.

Create role

In your Elastic Cloud instance, create a role for Fivetran. We recommend creating the role through Kibana.

From the Kibana home screen, go to Stack Management and click Roles (under Security).
In the top right corner of the screen, click Create Role.
In the Cluster privileges section, enter monitor.
Go to the Index privileges section.
- In the Indices field, enter the indices that you want Fivetran to have access to. You can either enter * to allow Fivetran to access all indices or list specific index names.
- In the Privileges field, enter read, view_index_metadata, and monitor.
Click Create role.

Create user

In your Elastic Cloud instance, create a user for Fivetran's exclusive use. We recommend creating the user through Kibana.

From the Kibana home screen, go to Stack Management and click Users (under Security).
In the top right corner of the screen, click Create User.
Enter your chosen username and password, then select the name of the role you created in the previous step.
Click Create user.

API key-based authentication

For an authentication based on an API key, you need to create and provide your API key to connect your Elastic Cloud instance to Fivetran.

Create an API key

In your Elastic Cloud instance, create an API key for Fivetran. We recommend creating the API key through Kibana.

From the Kibana home screen, go to Stack Management and click API keys (under Security).
In the top right corner of the screen, click Create API key.
In the Name field, replace fivetran-apikey with memorable API key name of your choice.
Click on Restrict privileges.
- For the Cluster parameter, add monitor as a cluster privilege.
- For the Indices parameter, add the indices that you want Fivetran to have access to. You can either add * to allow Fivetran to access all indices or list specific index names.
- For the Privileges parameter, add read, view_index_metadata, and monitor as index privileges.
- API keys are created on behalf of a user, so the resultant permissions would be an intersection of API keys permissions and authenticated user's permissions. Make sure the user has the required permissions.
Click Create API key.

Finish Fivetran configuration

In the connector setup form, enter a destination schema prefix. This prefix applies to each replicated schema and cannot be changed once your connector is created.
In the Host field, enter your Elasticsearch instance host's IP (e.g., 1.2.3.4) or domain (e.g., your.server.com).
Enter your Elasticsearch instance's port number.
Choose your authentication method. If you selected Use API key, provide the API key created in Step 2. If you selected Grant user access, provide the following information:
- Fivetran-specific user that you created in Step 2.
- Password for the Fivetran-specific user that you created in Step 2.
Choose your connection method. If you selected Connect via an SSH tunnel, copy or make a note of the Public Key and add it to the authorized_keys file while configuring the SSH tunnel, and provide the following information:
- SSH hostname (do not use a load balancer's IP address/hostname)
- SSH port
- SSH user
Click Save & Test. Fivetran tests and validates our connection to your Elastic Cloud database. Upon successful completion of the setup tests, you can sync your data using Fivetran.

Setup tests

Fivetran performs the following tests to ensure that we can connect to your Elastic Cloud instance and that it is properly configured:

The Connecting to SSH Tunnel Test validates the SSH tunnel details you provided in the setup form. It generates a pop-up window where you must verify the SSH fingerprint. It then checks that we can connect to your database using the SSH Tunnel. (We skip this test if you are connecting directly.)
The Validating Elasticsearch Connection Test validates the database credentials you provided in the setup form and checks that we can connect to the host.
The Validating Certificate Test validates the certificate you want Fivetran to use. It generates a pop-up window where you must choose the certificate. It then checks that we can connect to your database using TLS. (We skip this test if you are connecting using an SSH tunnel and did not choose to require TLS.)
The Validating Database Type Test checks that your database type matches the connector type. This test will throw a warning if we are unable to validate the database type.
The Validating Elasticsearch Version Test verifies that your Elasticsearch version is one of the versions that we support. This test will fail if we don't support the version.
The Validating User Privileges Test validates that the Fivetran user has the necessary privileges to perform a sync.

NOTE: The tests may take a few minutes to finish running.