Proxy Agent

Prerequisites:

• Proxy Agent version 1.1.4 or later (bundled with High-Volume Agent version 6.1.0_90 and later).
• The operating system user that runs the Proxy Agent service must have read access to the KeyStore file.
  • On Linux, the Proxy Agent service is started by root, so the KeyStore file must be readable by root. When generating the KeyStore, run the openssl command using sudo to ensure the file is created with the correct ownership.
  • On Windows, the Proxy Agent service runs under the LocalSystem account unless a different service user is selected during installation. Ensure that whichever account is used to run the service has read access to the KeyStore file.
• OpenSSL version 1.0.1 or later installed (which is required to convert PEM files into a PKCS12 KeyStore).

Perform the following steps to prepare and configure the KeyStore:

1. Extract the certificate and private key from the generated config.json file into PEM files. Run the following commands in the directory where the config.json file is located:

   Ensure you have jq installed on your system. If not, follow the instructions in the jq installation guide.

   jq -r .client_cert < config.json > certificate.pem
   jq -r .client_private_key < config.json > private_key.pem
   

2. Convert the PEM certificate and private key into a PKCS12 KeyStore file.

   OpenSSL is required to convert the PEM certificate and private key into a PKCS12 KeyStore file. Ensure that OpenSSL version 1.0.1 or later is installed. Earlier versions do not support the SHA-256 algorithms required for PKCS12 keystores. If OpenSSL is not installed already, install it using either your operating system’s package manager or by downloading the source from the OpenSSL project website.

   • Linux:

     sudo openssl pkcs12 -export \
       -in certificate.pem \
       -inkey private_key.pem \
       -out keystore.p12 \
       -name orchestrator.fivetran.com \
       -passout pass:<your_secure_password>
     

   • Windows:

     openssl pkcs12 -export -in certificate.pem -inkey private_key.pem -out keystore.p12 -name orchestrator.fivetran.com -passout pass:<your_secure_password>
     

3. Set permissions on the KeyStore file so the Proxy Agent service can read it.

   The KeyStore file must be readable by the operating system user that runs the Proxy Agent service.

   • Linux:
     Since the Proxy Agent service is started by root, creating the KeyStore file with sudo (as shown in the previous step) ensures that root can read the file.

   • Windows:
     Set file permissions based on the account used to run the Proxy Agent service.

     Replace <path_to_keystore> with the full path to your KeyStore file and <ServiceUserName> with the Windows account configured for the Proxy Agent service.

     • LocalSystem account (default):

       icacls <path_to_keystore> /grant:r "NT AUTHORITY\SYSTEM":R "BUILTIN\Administrators":F /C
       

     • Custom service user:

       icacls <path_to_keystore> /grant:r "<ServiceUserName>":R "BUILTIN\Administrators":F /C
       

4. (Optional) Validate that the KeyStore was created correctly using the JRE bundled with the Proxy Agent:

   • Linux:

     $HVR_HOME/jre/bin/keytool -list -keystore keystore.p12 -storetype PKCS12 -storepass <your_secure_password>
     

   • Windows:

     %HVR_HOME%\jre\bin\keytool -list -keystore keystore.p12 -storetype PKCS12 -storepass <your_secure_password>
     

     Expected output:

     Keystore type: PKCS12
     Keystore provider: SUN
     
     Your keystore contains 1 entry
     
     orchestrator.fivetran.com, Jan 15, 2025, PrivateKeyEntry,
     Certificate fingerprint (SHA-256): AA:BB:CC:...
     

5. Remove the embedded certificate fields from your configuration file. This ensures that the Proxy Agent relies only on the KeyStore for the certificate and private key.

   For Windows EXE installations, the installer requires embedded certificates for validation. Do not remove the client_cert and client_private_key fields before installation. Remove these fields after the installation completes, as described in Configure KeyStore settings for Windows EXE installation.

   Before (embedded certificates):

   {
     "agent_id": "your_agent_id",
     "auth_token": "...",
     "orchestrator_host": "orchestrator.fivetran.com",
     "client_cert": "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----\n",
     "client_private_key": "-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----\n"
   }
   

   After (using KeyStore):

   {
     "agent_id": "your_agent_id",
     "auth_token": "...",
     "orchestrator_host": "orchestrator.fivetran.com"
   }
   

• Using Telnet:

   telnet orchestrator.fivetran.com 443
  

• Using Netcat:

   nc orchestrator.fivetran.com 443 -v
  

Replace <proxy server hostname> with the hostname for the appropriate region:

• Using Telnet:

   telnet primary.<proxy server hostname> 443
  

• Using Netcat:

   nc primary.<proxy server hostname> 443 -v
  

When installing the Proxy Agent as a service, the user who installs the Proxy Agent must have permission to manage Windows services. We recommend that you install the Proxy Agent as an Administrator user.

1. Run the downloaded .exe file (e.g. fivetran-6.1.0_23-hub_and_agent-windows-x64-64bit_ga_patch-setup.exe).

2. In the installation wizard dialog, click Next.

   

3. Read the License Agreement, select I accept the agreement and click Next.

   

4. Specify the installation directories and click Next.

   

5. Specify the name for the program folder and click Next.

   

6. Select the role of the installation:

   • Proxy Agent to install the Proxy Agent

     

7. Paste the Proxy Agent settings from the configuration file (config.json) you've generated and click Next.

   The configuration file contains the settings and authentication parameters required to establish secure communication between the Proxy Agent and Fivetran.

   

8. Select the user account for running the Proxy Agent service.

   • If Specified user is selected, enter values for User, Password, Confirm Password and click Next.

     

   • If Local System account is selected, click Next.

     

9. Click Next to initiate the installation.

   

10. Click Finish to start the Proxy Agent. Once installed, the Proxy Agent service appears running in Windows Services.

    

11. (Optional) If you are using a KeyStore for certificate management, configure the Proxy Agent to load the KeyStore by setting the required JVM system properties.

    Expand for instructions

    
    

    Configure KeyStore settings for Windows EXE installation

    The Windows EXE installer creates the Proxy Agent as a Windows service using prunsrv.exe. The installer requires embedded certificates during installation for validation. After installation completes, you must remove the embedded certificates from the Proxy Agent configuration and then configure the service to load credentials from the KeyStore.

    1. Stop the Proxy Agent service:

       net stop "Fivetran Proxy Agent"
       

    2. Remove the embedded certificate fields from the Proxy Agent configuration file.

       a. Open the %HVR_CONFIG%\proxy\proxyagent.conf file.

       b. Remove the client_cert and client_private_key fields.

       Before (embedded certificates):

       {
       "agent_id": "your_agent_id",
       "auth_token": "...",
       "orchestrator_host": "orchestrator.fivetran.com",
       "client_cert": "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----\n",
       "client_private_key": "-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----\n"
       }
       

       After (using KeyStore):

       {
       "agent_id": "your_agent_id",
       "auth_token": "...",
       "orchestrator_host": "orchestrator.fivetran.com"
       }
       

       c. Save the file after removing the fields.

    3. Identify the actual service name created by the installer:

       sc query | findstr /i "FivetranProxy"
       

       Note the service name in the output. It has the format FivetranProxyPrunsrv_<HASH> (for example, FivetranProxyPrunsrv_ABCD1234).

    4. Update the service to include the KeyStore JVM system properties:

       "%HVR_HOME%\proxy\prunsrv.exe" //US//FivetranProxyPrunsrv_<HASH> ++JvmOptions "-Djavax.net.ssl.keyStore=<path_to_keystore>" ++JvmOptions "-Djavax.net.ssl.keyStorePassword=<your_secure_password>"
       
       

       Replace:

       • FivetranProxyPrunsrv_<HASH> with the service name you identified
       • <path_to_keystore> with the full path to your KeyStore file
       • <your_secure_password> with the KeyStore password
       Using ++JvmOptions appends options without removing existing JVM configuration required by the Proxy Agent.

    5. Start the Proxy Agent service:

       net start "Fivetran Proxy Agent"
       

    6. Check the Proxy Agent logs to confirm that the KeyStore was loaded successfully. By default, the log file is located at %HVR_CONFIG%\proxy\log\agent-out.log. Look for entries similar to:

       INFO: Certificates not found in config, attempting to load from KeyStore
       INFO: Using KeyStore from javax.net.ssl.keyStore: <path_to_keystore>
       INFO: Successfully loaded certificates from KeyStore for alias: orchestrator.fivetran.com
       

Perform the following steps in the user account that will be used for operating the Proxy Agent:

1. Configure the environment variables HVR_HOME, HVR_CONFIG, and HVR_TMP for your operating system using command setx or set. Each of these environment variables should be pointed to the installation directories - hvr_home, hvr_config, and hvr_tmp:

   Environment variables set using setx command are available in the future command windows only and the environment variables set using set command are available in the current command window only.

   setx HVR_HOME C:\fivetran\hvr_home
   setx HVR_CONFIG C:\fivetran\hvr_config
   setx HVR_TMP C:\fivetran\hvr_tmp
   

   set HVR_HOME=C:\fivetran\hvr_home
   set HVR_CONFIG=C:\fivetran\hvr_config
   set HVR_TMP=C:\fivetran\hvr_tmp
   

   Also, add the executable directory path (e.g C:\fivetran\hvr_home\bin) to the environment variable PATH.

   setx PATH "%PATH%C:\fivetran\hvr_home\bin"
   

   set PATH=%PATH%;C:\fivetran\hvr_home\bin
   

   Alternatively, environment variables can be configured using Windows GUI.

   1. Navigate to Control Panel ▶ System and Security ▶ System ▶ Advanced system settings

      Alternatively, use the command sysdm.cpl to open System Properties.

   2. In the Advanced tab, click Environment Variables...

      

   3. In section System variables or User Variables for user_name, click New.

      1. Enter Variable name (e.g, HVR_HOME) and Variable value (e.g, C:\fivetran\hvr_home).

         

      2. Click OK.

      3. Repeated the above steps for each environment variable.

   4. Add the executable directory path to the environment variable Path.

      1. In section System variables or User Variables for user_name, from the list of variables, select Path and click Edit....

      2. Click New and enter the path for the Proxy Agent executable.

         

      3. Click OK.

2. Create the installation directory - hvr_home (e.g. C:\fivetran\hvr_home):

   md %HVR_HOME%
   

   • Other directories (hvr_config and hvr_tmp) will be created automatically as needed.
   • hvr_home is regarded a read-only directory.

3. Uncompress (unzip) the installation file (e.g. fivetran-6.1.0_23-hub_and_agent-windows-x64-64bit_ga_patch.zip) into the hvr_home directory:

   cd %HVR_HOME%
   C:\fivetran\hvr_home>tar -xf C:\Users\Admin\Downloads\fivetran-6.1.0_23-hub_and_agent-windows-x64-64bit_ga_patch.zip
   

   Alternatively, files can be uncompressed using the 'Extract All' option in Windows GUI.

   

4. Paste the Proxy Agent settings from the configuration file (config.json) you've generated in Step 6 to the proxyagent.conf file located in the HVR_CONFIG/proxy directory.

   The configuration file contains the settings and authentication parameters required to establish secure communication between the Proxy Agent and Fivetran.

   If the proxyagent.conf file is missing, create the file manually.

5. Run the following command to validate the Proxy Agent settings:

   %HVR_HOME%/jre/bin/java -jar %HVR_HOME%/proxy/proxyagent.jar -v %HVR_CONFIG%/proxy/proxyagent.conf
   

6. Create a .bat file (e.g. install_and_run_proxy_service.bat) with the following contents.

    FOR /F "delims=" %%i IN ('CALL "%HVR_HOME%/bin/hvr" hvrhubserversvc -s') DO SET HvrHubServerSvcOutput=%%i
    REM We use a normalized HVR_CONFIG path hash to guarantee a unique service name
    SET HubConfigHash=%HvrHubServerSvcOutput:~13,8%
   
    REM In the CLI below use "DomainName\UserName" for --ServiceUser and specify --ServicePassword parameter if needed
    REM Adjust the other parameters as necessary
    REM Ensure to have unique service name (in //IS//<ServiceName>) and display name 
    %HVR_HOME%/proxy/prunsrv.exe //IS//FivetranProxy_%HubConfigHash% ^
       --StartParams "%HVR_CONFIG%/proxy/proxyagent.conf" ^
       --ServiceUser "LocalSystem" ^
       --DisplayName "Fivetran Proxy Agent [%HubConfigHash%]" ^
       --Description "Fivetran Proxy Agent installed into %HVR_HOME%\proxy" ^
       --Jvm "%HVR_HOME%/jre/bin/server/jvm.dll" ^
       --JavaHome "%HVR_HOME%/jre" ^
       --Classpath "%HVR_HOME%/proxy/proxyagent.jar" ^
       --StartPath "%HVR_HOME%/proxy/" ^
       --Startup=auto ^
       --StartMode=jvm ^
       --StartClass=com.fivetran.proxy.agent.ProxyAgent ^
       --StartMethod=main ^
       --StopMode=jvm ^
       --StopClass=com.fivetran.proxy.agent.ProxyAgent ^
       --StopMethod=stop ^
       --JvmOptions "-XX:+HeapDumpOnOutOfMemoryError" ^
       --StdOutput "%HVR_CONFIG%/proxy/logs/agent-out.log" ^
       ++Environment "HVR_CONFIG=%HVR_CONFIG%" ^
       ++Environment "HVR_HOME=%HVR_HOME%" ^
       ++Environment "HVR_TMP=%HVR_TMP%"
   
     %HVR_HOME%/proxy/prunsrv.exe //ES//FivetranProxy_%HubConfigHash%
   

7. Run the .bat file to configure and start the Proxy Agent:

   install_and_run_proxy_service.bat
   

8. (Optional) If you are using a KeyStore for certificate management, configure the Proxy Agent to load the KeyStore by setting the required JVM system properties.

   Expand for instructions

   
   

   Configure KeyStore settings for Windows ZIP installation (optional)

   1. Stop the Proxy Agent service:

      %HVR_HOME%\bin\hvrproxy.bat stop
      

   2. Update the Proxy Agent service definition to include the required JVM system properties for loading the KeyStore. Open the install_and_run_proxy_service.bat file you created earlier and modify the --JvmOptions line:

      --JvmOptions "-XX:+HeapDumpOnOutOfMemoryError; -Djavax.net.ssl.keyStore=<path_to_keystore>; -Djavax.net.ssl.keyStorePassword=<your_secure_password>"
      

   3. Rerun the updated install script to update the service definition:

      install_and_run_proxy_service.bat
      

   4. Start the Proxy Agent service:

      %HVR_HOME%\bin\hvrproxy.bat start
      

   5. Check the Proxy Agent logs to confirm that the KeyStore was loaded successfully. By default, the log file is located at %HVR_CONFIG%\proxy\log\agent-out.log. If you have configured a custom log directory using the log_folder_path setting in proxyagent.conf (located in %HVR_CONFIG%\proxy), the logs are written to that directory instead (for example, <log_folder_path>\agent-out.log). Look for entries similar to:

      INFO: Certificates not found in config, attempting to load from KeyStore
      INFO: Using KeyStore from javax.net.ssl.keyStore: <path_to_keystore>
      INFO: Successfully loaded certificates from KeyStore for alias: orchestrator.fivetran.com
      

Perform the following steps as the user that will be used for operating Proxy Agent:

1. Configure the environment variables HVR_HOME, HVR_CONFIG, and HVR_TMP for your operating system. Each of these environment variables should be pointed to the installation directories - hvr_home, hvr_config, and hvr_tmp.

   export HVR_HOME=/home/fivetran/hvr_home 
   export HVR_CONFIG=/home/fivetran/hvr_config
   export HVR_TMP=/home/fivetran/hvr_tmp
   

   Also, add the executable directory path to the environment variable PATH.

   PATH=$PATH:$HVR_HOME/bin
   

2. Add the environment and the executable directory path into the startup file (e.g. .profile).

   export HVR_HOME=/home/fivetran/hvr_home
   export HVR_CONFIG=/home/fivetran/hvr_config
   export HVR_TMP=/home/fivetran/hvr_tmp
   export PATH=$PATH:$HVR_HOME/bin
   

3. Create the installation directory - hvr_home using the following commands:

   umask 022
   

   mkdir $HVR_HOME
   

   umask 022 is used so that the files and directories created in the following commands are readable by everyone (other Linux users and groups), but only writable by the owner. Other directories (HVR_CONFIG and HVR_TMP) will be created automatically as needed. The HVR_HOME directory is regarded as read-only.

4. Uncompress the installation file (e.g. fivetran-6.1.0_23-hub_and_agent-linux_glibc2.17-x64-64bit_ga.tar.gz) into the HVR_HOME directory:

   cd $HVR_HOME
   

   tar xzf /tmp/hvr-6.1.0_23-hub_and_agent-linux_glibc2.17-x64-64bit_ga.tar.gz
   

   Once installed, the jre and proxy folders are created in your HVR_HOME directory.

5. Create a new directory for the proxyagent.conf file in hvr_config (/home/fivetran/hvr_config/proxy). Paste the Proxy Agent settings from the configuration file (config.json) you've generated to the proxyagent.conf file located in the newly created directory.

   The configuration file contains the settings and authentication parameters required to establish secure communication between the Proxy Agent and Fivetran.

6. Run the following command to verify if the Linux machine is ready for the setup. The output of the command should be systemd.

   ps -p 1 -o comm=
   

7. Update the fivetran_proxy.service file in the /etc/systemd/system directory with the correct path to java.jar, proxyagent.jar, and proxyagent.conf files.

   If missing, create the fivetran_proxy.service file manually. The contents of the file should be as follows:

   [Unit]
   Description=Fivetran Proxy Agent #<agent_id>
   
   [Service]
   
   Type=simple
   Environment="HVR_HOME=/home/fivetran/hvr_home"
   Environment="HVR_CONFIG=/home/fivetran/hvr_config"
   Environment="HVR_TMP=/home/fivetran/hvr_tmp"
   
   ExecStart=${HVR_HOME}/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${HVR_CONFIG}/proxy/ -jar ${HVR_HOME}/proxy/proxyagent.jar ${HVR_CONFIG}/proxy/proxyagent.conf
   
   # Restart this service to after a crash
   Restart=always
   
   # The number of seconds to wait before attempting a restart
   RestartSec=5s
   
   [Install]
   WantedBy=multi-user.target
   

8. Execute the following commands as the root user to start the Proxy Agent service.

   a. Enable the service:

   systemctl enable fivetran_proxy.service
   

   b. Start the service:

   systemctl start fivetran_proxy.service
   

   c. Verify the status of the service:

   systemctl status fivetran_proxy.service
   

9. (Optional) If you are using a KeyStore for certificate management, configure the Proxy Agent to load the KeyStore by setting the required JVM system properties.

   Expand for instructions

   
   

   Configure KeyStore settings for Linux installation (optional)

   1. Update the systemd service definition to include the required JVM system properties for loading the KeyStore. Edit the /etc/systemd/system/fivetran_proxy.service file and add the following line under the [Service] section:

      Environment="JAVA_OPTS=-Djavax.net.ssl.keyStore=/path/to/keystore.p12 -Djavax.net.ssl.keyStorePassword=<your_secure_password>"
      

   2. Reload the systemd daemon and restart the Proxy Agent service:

      sudo systemctl daemon-reload
      sudo systemctl restart fivetran_proxy.service
      

   3. Check the Proxy Agent logs to confirm that the KeyStore was loaded successfully. By default, the log file is located at %HVR_CONFIG%/proxy/log/agent-out.log. If you have configured a custom log directory using the log_folder_pathsetting in proxyagent.conf (located in $HVR_CONFIG/proxy), the logs are written to that directory instead (for example, <log_folder_path>/agent-out.log). Look for entries similar to:

      INFO: Certificates not found in config, attempting to load from KeyStore
      INFO: Using KeyStore from javax.net.ssl.keyStore: <path_to_keystore>
      INFO: Successfully loaded certificates from KeyStore for alias: orchestrator. fivetran.com
      

If your Proxy Agent is installed on a Windows system, you can set up a recovery configuration for the agent by performing the following steps:

1. Go to Control Panel > Administrative Tools > Computer Management > Services and Applications > Services or use the services.msc command to open the Services console.

2. In the Services console, locate the Fivetran Proxy Agent service. Ensure the service description matches the installation directory.

3. Right-click the service and select Properties.

4. Go to the Recovery tab.

5. Set your preferred recovery options for the failure scenario.

   

1. Download the latest High-Volume Agent Installer from the downloads page.

2. Run the Installer

The Proxy Agent installed using a ZIP file or on Linux requires manual upgrade steps. Perform the following steps:

1. Download the latest High-Volume Agent for the desired operating system from the downloads page.

2. Uninstall the existing proxy agent using the Uninstall Proxy Agent instructions.

3. Follow the instructions for installing the Proxy Agent using the appropriate section of the installation instructions.

The steps to uninstall the Proxy Agent depend on whether you used an EXE file (Installer) or a ZIP file to install it.

Uninstall Proxy Agent that was installed using an EXE file

1. Navigate to the uninstall folder in the Proxy Agent installation directory %HVR_HOME% (for example, C:\fivetran\hvr_home\uninstall).

2. Double-click the uninstall.exe file to initiate the uninstallation process.

Uninstall Proxy Agent that was installed using a ZIP file

1. Create a .bat file (for example, delete_proxy_agent_service.bat) with the following contents:

      FOR /F "delims=" %%i IN ('CALL "%HVR_HOME%/bin/hvr" hvrhubserversvc -s') DO SET HvrHubServerSvcOutput=%%i
      REM We use a normalized HVR_CONFIG path hash to guarantee a unique service name
      SET HubConfigHash=%HvrHubServerSvcOutput:~13,8%
   
      %HVR_HOME%/proxy/prunsrv.exe //DS//FivetranProxy_%HubConfigHash%
   

2. Run the .bat file as an Administrator to remove all Proxy Agent service entries.

1. Stop the Proxy Agent service:

    sudo systemctl stop fivetran_proxy.service
   

2. Disable the service from auto-starting at boot:

    sudo systemctl disable fivetran_proxy.service 
   

3. Remove the service configuration file:

    sudo rm /etc/systemd/system/fivetran_proxy.service
   

4. Reload the systemd manager configuration:

    sudo systemctl daemon-reload
   

5. Clear the systemd state for any failed services:

    sudo systemctl reset-failed