Oracle Setup Guide
Follow these instructions to replicate your generic Oracle database to your destination via Fivetran.
To connect your Oracle database to Fivetran, you need:
- Oracle 11g or above
- IP (e.g. 220.127.116.11) or host (your.server.com)
- Port (Usually
1521for unencrypted connections and
2484for encrypted connections using SSL/TLS)
Decide whether to connect to your destination directly or connect using an SSH tunnel. If your Oracle database is version 12.1 or below, you'll need to connect using an SSH tunnel.
- To connect using an SSH tunnel, follow these instructions.
- To connect directly, configure your firewall and/or other access control systems to allow incoming connections to your Oracle database host and port (usually
1521) from Fivetran's IP.
Connect to your Oracle database as an admin user.
Execute the following SQL commands to create a user for Fivetran and grant it permission to connect to your database. Replace
<password>with a password of your choice:
CREATE USER FIVETRAN IDENTIFIED BY <password>; GRANT CREATE SESSION TO FIVETRAN;
Note: Usernames in Oracle are case sensitive.
fivetranis not the same user as
Grant read-only access
Once you've created the
FIVETRANuser, grant it
SELECTpermission for each schema and table you would like to sync:
GRANT SELECT ON "<schemaA>"."<tableA>" TO FIVETRAN; GRANT SELECT ON "<schemaA>"."<tableB>" TO FIVETRAN; GRANT SELECT ON "<schemaB>"."<tableC>" TO FIVETRAN;
Alternatively, you can grant access to all tables:
GRANT SELECT ANY TABLE TO FIVETRAN;
FIVETRANuser access to the
GRANT SELECT ON DBA_EXTENTS TO FIVETRAN; GRANT SELECT ON DBA_TABLESPACES TO FIVETRAN; GRANT SELECT ON DBA_SEGMENTS TO FIVETRAN;
We use these views to optimize our initial import queries.
Note: Oracle database defaults to using uppercase, unless values are surrounded by double quotes.
Choose incremental update mechanism
To keep your data up to date after the initial sync, we use one of two built-in Oracle technologies: Flashback and LogMiner. Both Flashback and LogMiner keep a record of recent data changes, which allows Fivetran to update only the data that has changed since our last sync.
Choose either Flashback or LogMiner as your incremental update mechanism.
- To enable Flashback, proceed to the next section.
- To enable LogMiner, skip ahead to the LogMiner section.
Flashback PRIVATE PREVIEW
Flashback uses Automatic Undo Management to record all committed changes for each included table. To enable Flashback, follow these steps:
FIVETRANuser access to additional system views:
GRANT SELECT ON DBA_FLASHBACK_ARCHIVE TO FIVETRAN; GRANT SELECT ON DBA_FLASHBACK_ARCHIVE_TABLES TO FIVETRAN; GRANT SELECT ON V_$SYSTEM_PARAMETER TO FIVETRAN; GRANT SELECT ON V_$DATABASE TO FIVETRAN;
Create a tablespace with enough space to store about 7 days’ worth of database changes.
CREATE TABLESPACE <tablespace> DATAFILE SIZE 1G AUTOEXTEND ON MAXSIZE <maximum size>;
Create a Flashback Data Archive with at least 1 day of retention (Fivetran recommends 7 days).
CREATE FLASHBACK ARCHIVE <archive> TABLESPACE <tablespace> RETENTION 7 DAY;
Add all tables you want to sync to the archive.
ALTER TABLE "<schema>"."<table>" FLASHBACK ARCHIVE <archive>;
FIVETRANuser permission to run Flashback queries on the tables you want to sync. You can either do this per table:
GRANT FLASHBACK ON "<schema>"."<table>" TO FIVETRAN;
or for all tables:
GRANT FLASHBACK ANY TABLE TO FIVETRAN;
Add an index to FDA-generated history tables (Optional)
Flashback queries are significantly more efficient with an index on the history tables generated by Flashback Data Archive. The addition of this index can make your sync significantly faster and more resource-efficient, especially if your database is large or has a high change volume. To add indices, follow these steps:
Find the system-generated history tables for each of your Flashback-enabled tables by running the following query:
SELECT TABLE_NAME, ARCHIVE_TABLE_NAME FROM DBA_FLASHBACK_ARCHIVE_TABLES;
For each of the
ARCHIVE_TABLE_NAMEs returned in the previous query, add an index on the
CREATE INDEX <index-name> ON <archive-table-name> (STARTSCN);
LogMiner uses Archived Redo Log files on your Oracle database to get a list of modified tables and determine which rows need to be updated incrementally in the destination. Therefore, we require the following configuration to be present on your Oracle database:
If ARCHIVELOG mode is not enabled on your database, enable ARCHIVELOG mode:
Note: Enabling ARCHIVELOG mode requires the Oracle instance to be taken offline for a brief period. To learn more, see Oracle's documentation.
SHUTDOWN IMMEDIATE; STARTUP MOUNT; ALTER DATABASE ARCHIVELOG; ALTER DATABASE OPEN;
Configure Oracle RMAN to retain Backups and Archive Logs for at least 24 hours (Fivetran recommends 7 days):
Note: Fivetran must have a minimum of 3 hours worth of log data to pull from. You cannot finish setting up your connector until 3 hours after RMAN has been configured to retain Archive Logs.
RMAN> CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;
Fivetran recommends setting
DB_RECOVERY_FILE_DEST_SIZEto a value that agrees with your available disk space because Expired and Obsolete Log and Backup files can quickly fill your disk. For more information, see Oracle's Documentation.
Enable supplemental logging for each table you plan to sync.
If you plan to only sync specific tables, run the following query for each table you want to include:
ALTER TABLE "<schema>"."<table>" ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
If you plan to sync all tables in your database, run the following query:
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
For each table where the primary key is expected to change, run the following query:
ALTER TABLE "<schema>"."<table>" ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS
If you don't configure the logging data correctly, you will receive a warning when Fivetran encounters a primary key change. The warning will give you customized instructions.
Grant permissions for running LogMiner to the
GRANT SELECT ON SYS.V_$DATABASE TO FIVETRAN; GRANT SELECT ON SYS.V_$ARCHIVED_LOG TO FIVETRAN; GRANT SELECT ON SYS.V_$LOGMNR_CONTENTS TO FIVETRAN; GRANT EXECUTE ON DBMS_LOGMNR TO FIVETRAN; GRANT EXECUTE ON DBMS_LOGMNR_D TO FIVETRAN; GRANT SELECT ANY TRANSACTION TO FIVETRAN; GRANT EXECUTE_CATALOG_ROLE TO FIVETRAN;
Grant additional permissions for Oracle version 12:
GRANT LOGMINING TO FIVETRAN;