SAP ERP on HANA
SAP Enterprise Resource Planning (ERP) applications like SAP ECC and SAP S/4HANA manage customer finance, operations, supply chain, and HR processes. With the SAP ERP on HANA connector, Fivetran can replicate data from SAP ECC and S/4HANA systems running on a HANA platform.
SAP HANA is an in-memory, column-oriented, relational database management system. Fivetran replicates data from your SAP HANA source database and loads it into your destination.
You must have an Enterprise or Business Critical plan to use the SAP ERP on HANA connector.
Features
| Feature Name | Supported | Notes |
|---|---|---|
| Capture deletes | ||
| History mode | ||
| Custom data | ||
| Data blocking | ||
| Column hashing | ||
| Re-sync | ||
| Row filtering | ||
| API configurable | API configuration | |
| Priority-first sync | ||
| Fivetran data models | ||
| Private networking |
| |
| Authorization via API |
Supported deployment models
We support the SaaS and Hybrid deployment models for the connector.
You must have an Enterprise or Business Critical plan to use the Hybrid Deployment model.
Connection modes
The SAP ERP on HANA connector supports two connection modes:
- NetWeaver Connection — Connects through the SAP NetWeaver Application Server using RFC calls. Requires the Fivetran NetWeaver API to be installed in your SAP system.
- HANA Database Connection — Connects directly to the SAP HANA database via JDBC, bypassing the NetWeaver/ABAP layer. No NetWeaver API installation is required.
| NetWeaver Connection | HANA Database Connection | |
|---|---|---|
| SAP ABAP version | Unicode-compliant SAP ABAP 7.4 or later | Not required |
| HANA DB version | SAP HANA 2.0 SPS 03 or later | SAP HANA 2.0 SPS 03 or later |
| Fivetran component | Fivetran NetWeaver API installed on your SAP system | You must create the FIVETRAN_DB schema in the HANA database. We use this schema to store shadow tables for delete tracking. See the SAP ERP on HANA Setup Guide for instructions. |
| User account | SAP Communication or System user with required authorizations | Dedicated HANA database user with required privileges. See the SAP ERP on HANA Setup Guide for details. |
| Connection details | Host/IP address, system number (SYSNR), client number (CLIENT) | Host/IP address, HANA DB port, schema name |
| Data types | SAP ABAP types (from SAP ABAP Dictionary) | Native SAP HANA types |
| SAP ABAP Dictionary | Required | Not used |
| No-trigger sync mechanism | Supported | Not supported |
| SAP Archiving | Supported | Not supported |
| Unpack data | Supported | Not supported |
Known limitations
Table and view support:
- We support column-store and row-store transparent tables, as well as views. For the NetWeaver Connection mode, these are sourced from the SAP ABAP Dictionary. Avoid replicating base tables that point to proxy objects, such as compatibility views, because they may return incomplete or no data. Instead, sync the compatibility view directly. Learn more in Excluded objects.
- No support for parameterized views and projection views. Learn more in Excluded objects.
- No support for change data capture (CDC) in row store transparent tables and views. Learn more in Replication of views and row-store tables.
- No support for SAP BW Advanced DataStore Object (ADSO) tables.
Search and selection:
When selecting tables to sync, the following limitations apply. For more information, see Selecting data to sync:
- Each search term must include at least three alphanumeric characters.
- You can use the following special characters in your search terms:
*,?,_,-,., and/.
Schema and naming:
Each connection supports only one source schema. To replicate tables from multiple schemas, create a separate connection for each schema.
NetWeaver Connection only:
- Column and table names have limited support for special characters and lowercase characters.
- Schema names do not support special characters or lowercase characters. For example,
lowercase_schema,lowercase_SCHEMA, andlOwErCaSeScHeMaare all unsupported.
* Special characters include
(,&,@,*,$,|,%,~,-, and). Underscores are not considered special characters.
API and authorization:
- NetWeaver Connection: Authorization through API is supported only when using SSH tunneling.
Sync behavior:
- No support for the following DDL statements:
CREATE TABLEand mostALTER TABLEoperations. If such operations occur, the respective table must be manually re-synced. However,ADD COLUMNandDROP COLUMNoperations are supported. - No support for updating a primary key value. If a primary key changes in a source, we treat it as a new
INSERTin a destination. - History mode is supported, but we only capture record updates at the time of sync. Updates that occur between two syncs are not captured.
- No support for the following DDL statements:
Version requirements:
- NetWeaver Connection: We require Unicode-compliant SAP ABAP 7.4 or later. Learn more in Prerequisites in SAP ERP on HANA Setup Guide.
- Compatible with SAP HANA 2.0 SPS08 and supports the partitioning features available in this version.
Setup guide
Follow our step-by-step SAP ERP on HANA Setup Guide for specific instructions on how to set up your SAP ERP on HANA using the Fivetran connector.
Sync overview
NetWeaver Connection
The SAP ERP on HANA connector interfaces with the application layer of an SAP system. Within this application layer, you need to install the Fivetran NetWeaver API, which then pulls a full dump of all selected data from your SAP system. We then use our proprietary tracking mechanism to retrieve modified data at regular intervals.
HANA Database Connection
This mode connects directly to the SAP HANA database through JDBC, without routing through the NetWeaver application layer. We use SQL queries to read data directly from the HANA schema you specify during setup. You don't need to install the Fivetran NetWeaver API.
Syncing empty tables and columns
Fivetran can sync empty tables for your SAP ERP on HANA connection.
We can also sync empty columns. However, we require at least one row of data that includes this new column to be added. This is because we detect changes to a table's column cardinality when we see a row with a new or removed column during an update.
For more information, see our Features documentation.
Schema information
Fivetran replicates the exact schema and tables from your HANA source database to your destination according to our standard database update strategies. This ensures that the data in your destination is in a familiar format to work with.
When you connect to Fivetran and specify a source database, you also select a schema prefix. We map the SAP schema in your source database to your destination and prepend the destination schema name with the prefix you selected.
Fivetran-generated columns
Fivetran adds system columns to your destination tables to give you insight into your sync activity and row-level changes. The columns added depend on the sync mode, sync mechanism, whether archive deletes tracking is enabled, and table type.
Fivetran-generated columns are not included in the list of filterable columns when you apply table filters. For more information, see Filtering rows based on column values.
Columns by sync mode
Depending on the sync mode you use, we may add the following columns to destination tables:
- All sync modes:
_fivetran_synced(UTC TIMESTAMP) indicates the time when Fivetran last successfully synced the row. - Soft delete sync mode:
_fivetran_deleted(BOOLEAN) marks rows that were deleted in the source database. - History sync mode:
_fivetran_active(BOOLEAN),_fivetran_start(UTC TIMESTAMP), and_fivetran_end(UTC TIMESTAMP). These columns identify the active row version and preserve the lifetime of previous row versions.
Columns for tracking SAP archive deletes
If you enable archive deletes tracking in your SAP source, we add the _fivetran_sap_archived column to indicate the archived records.
Columns in No-trigger sync mechanism
When you use the No-trigger sync mechanism, we add the following column to column-store transparent tables:
_fivetran_id(LONG): A Fivetran-generated primary key column. This column replaces the original primary key constraint in the destination table.
In this configuration, column-store transparent tables are replicated in Live mode. We don't add _fivetran_deleted, _fivetran_active, _fivetran_start, _fivetran_end, and _fivetran_sap_archived columns to these tables. Instead, when rows are deleted in the source, we remove them permanently from the destination using hard deletes.
Columns for unpacked tables
For certain packed tables, such as STXL and MDTC, we may generate additional columns during unpacking. For more information, see Unpack data.
Columns for views and row-store transparent tables
We do not add the _fivetran_deleted, _fivetran_active, and _fivetran_sap_archived columns to views and row-store transparent tables.
For views specifically, we introduce the Fivetran-generated primary key column _fivetran_id (LONG).
For more information, see Replication of views and row-store tables.
Type transformation and mapping
As we extract your data, we match SAP data types to the types that Fivetran supports. Our system attempts to infer the types of any columns with data types that we do not recognize.
The following table illustrates how we transform your SAP data types (based on the SAP ABAP Dictionary) into Fivetran supported types:
| SAP Type | Fivetran Type | Fivetran Supported |
|---|---|---|
| ACCP | STRING | True |
| CHAR | STRING | True |
| CLNT | STRING | True |
| CUKY | STRING | True |
| CURR | BIGDECIMAL | True |
| DATS | STRING/LOCALDATE * | True |
| DEC | BIGDECIMAL | True |
| D16D | BIGDECIMAL | True |
| D16N | BIGDECIMAL | True |
| D16R | BINARY | True |
| D16S | BINARY | True |
| D34D | BIGDECIMAL | True |
| D34N | BIGDECIMAL | True |
| D34R | BINARY | True |
| D34S | BINARY | True |
| FLTP | DOUBLE | True |
| INT1 | SHORT | True |
| INT2 | SHORT | True |
| INT4 | INTEGER | True |
| INT8 | LONG | True |
| LANG | STRING | True |
| LCHR | STRING | True |
| LRAW | BINARY | True |
| NUMC | STRING | True |
| PREC | BIGDECIMAL | True |
| RAW | BINARY | True |
| RSTR | BINARY | True |
| QUAN | BIGDECIMAL | True |
| SSTR | STRING | True |
| STRG | STRING | True |
| TIMS | STRING | True |
| UNIT | STRING | True |
| VARC | - | False |
*The STRING or LOCALDATE type conversion depends on the Enable SAP date to LocalDate conversion toggle’s state in the connection setup form.
In HANA Database Connection mode, Fivetran reads native SAP HANA data types directly from the database. The following table illustrates how HANA data types are mapped to Fivetran types:
| HANA Type | Fivetran Type | Fivetran Supported |
|---|---|---|
| DATE | LOCALDATE | True |
| DAYDATE | LOCALDATE | True |
| SECONDDATE | LOCALDATETIME | True |
| TIMESTAMP | LOCALDATETIME | True |
| LONGDATE | LOCALDATETIME | True |
| TINYINT | SHORT | True |
| SMALLINT | SHORT | True |
| INTEGER | INTEGER | True |
| INT | INTEGER | True |
| BIGINT | LONG | True |
| SMALLDECIMAL | BIGDECIMAL | True |
| DECIMAL | BIGDECIMAL | True |
| DEC | BIGDECIMAL | True |
| NUMERIC | BIGDECIMAL | True |
| REAL | FLOAT | True |
| FLOAT | DOUBLE | True |
| DOUBLE | DOUBLE | True |
| BOOLEAN | BOOLEAN | True |
| VARCHAR | STRING | True |
| NVARCHAR | STRING | True |
| ALPHANUM | STRING | True |
| SHORTTEXT | STRING | True |
| TEXT | STRING | True |
| BINTEXT | STRING | True |
| TIME | STRING | True |
| SECONDTIME | STRING | True |
| CHAR | STRING | True |
| CLOB | STRING | True |
| NCLOB | STRING | True |
| VARBINARY | BINARY | True |
| BINARY | BINARY | True |
| BLOB | BINARY | True |
| ARRAY | JSON | True |
| ST_POINT | JSON | True |
| ST_GEOMETRY | JSON | True |
If we are missing an important data type that you need, reach out to support.
In some cases, when loading data into your destination, we may need to convert Fivetran data types into data types that are supported by the destination. For more information, see the individual data destination pages.
Data mapping workflows
Depending on the values, the data mapping for the connection can be in one of the following situations:
- The value can be parsed to a Fivetran data type - Fivetran takes the value and sends it to the destination in the appropriate data type.
- The value cannot be parsed to a Fivetran data type, and the column is not a primary key - Fivetran coerces the value to
nulland displays a warning. The sync continues. - The value cannot be parsed to a Fivetran data type, and the column is a primary key - The sync for the respective table ends with an error.
Schema changes
New tables are not automatically added. Use the Schema tab to add tables as needed.
Selecting data to sync
The SAP ERP on HANA connector does not auto-ingest your SAP catalog because SAP systems often contain hundreds of thousands of dictionary tables. You must explicitly select the tables or views you want to sync during the Select data to sync step when setting up your connection.
The following SAP object types are available for selection:
Column-store transparent tables
Row-store transparent tables
Views
For more information on how views and row-store transparent tables are handled during replication, see our Replication of views and row-store tables documentation.
New tables are not added automatically. You can add or remove tables or views at any time on your connection’s Schema tab; changes take effect on the next sync.
For the NetWeaver Connection mode, you can also restrict which tables and views appear for selection using the Source tables list parameter in Fivetran's SAP configuration screen. For more information, see our Limiting table availability documentation.
Limiting table availability
This feature is only available on NetWeaver Connection mode and for the Fivetran NetWeaver API version 1900230 and higher.
You can control which tables are available for selection in the Schema tab by defining the Source tables list in the Fivetran's SAP configuration screen accessible through the SAP transaction /N/FIVETRAN/CONFIG. For more information, see Configure Fivetran in SAP.
- If the list is empty (default), all supported tables and views are available for search and selection.
- If the list includes specific tables (for example,
T000andBSEG), only those tables will be visible in the Schema tab.
To stop syncing a previously added table that may already contain data in your destination, remove it from the Source tables list. We will disable the table in the Schema tab during the next sync and prevent further replication.
Example scenarios:
Here are some examples to illustrate how table filtering works:
Scenario 1 - No Filtering: If no tables are added to the Source tables list in the Fivetran's SAP configuration screen, all supported tables and views will be available for replication.
Scenario 2 - Adding new tables: The Schema tab contains tables
T000andT001. If a user adds tablesT000,T001,T002, andBSEGto the Source tables list, the existing tables (T000andT001) can still be replicated. However, only the newly added tables (T002andBSEG) will be available for searching and adding for sync in the Schema tab.Scenario 3 - Replacing existing tables: The Schema tab contains tables
T000andT001. If a user adds only tableBSEGto the Source tables list, the existing tables (T000andT001) will be automatically removed from the connection's Schema tab. Only tableBSEGwill be available for searching and adding for sync in the Schema tab.Scenario 4 - Removing and adding tables: If the Schema tab contains tables
T000andT001, and these tables are no longer permitted because they are not specified in the Source tables list, Fivetran will gray them out in the Schema tab and will not sync them. If you re-add tablesT000andT001, Fivetran will sync them and gray them out during the next sync cycle.
Filtering rows based on column values
You can apply row-level filters to replicate only the data you need from a specific table. For more information on creating, editing, or deleting filters, including supported operators and data types, see our Row Filtering documentation.
The Fivetran-generated columns cannot be used in filters because they are not part of the original source tables.
When applying filters to string columns, be aware that the connector does not ignore trailing spaces. For accurate filtering, ensure your values match the full content of the source field.
Initial sync
The initial sync can be performed once the connection to the SAP ERP on HANA database is established. For the NetWeaver Connection mode, the Fivetran NetWeaver API manages all data transport during this process. For the HANA Database Connection mode, Fivetran reads data directly from the HANA database via JDBC.
Updating data
Fivetran performs incremental updates of any new or modified data from your source database. The update behavior depends on two independent settings: the sync mechanism selected during connection setup and the sync mode configured per table.
Sync mechanisms
The sync mechanism determines how Fivetran detects changes, including deletes, in your SAP system. You select the mechanism once during connection setup; it cannot be changed after the connection setup is saved.
Default sync mechanism
Fivetran creates delete-capture triggers and shadow tables in your SAP HANA database for each column-store transparent table. Column-store transparent tables retain their original primary keys and support soft delete and history sync modes.
No-trigger sync mechanism
This sync mechanism is available for the NetWeaver Connection mode only. The HANA Database Connection mode does not support the No-trigger sync mechanism.
Fivetran detects changes using queries only. No triggers or shadow tables are created in your SAP HANA database. Column-store transparent tables are replicated in Live mode with a Fivetran-generated _fivetran_id (LONG) column replacing the original primary key.
| Sync mechanism | Default | No-trigger |
|---|---|---|
| Delete capture method | Triggers and shadow tables | Snapshots (query-based) |
| Creates triggers or shadow tables in SAP HANA | Yes | No |
| Available sync modes | Soft delete, History | Live only |
| Primary key for column-store tables | Original | Fivetran-generated _fivetran_id |
| Additional disk space required | No | Yes (for snapshots) |
The No-trigger sync mechanism requires additional disk space for storing internal snapshots. Take this into account when planning your Hybrid Deployment setup.
Sync modes
The sync mode controls how changes are represented in your destination. You can configure the sync mode per table on the Schema tab.
Soft delete and history modes are available only when using the Default sync mechanism. When using the No-trigger sync mechanism, column-store transparent tables are always replicated in Live mode.
Soft delete mode
- An
INSERTin the source table generates a new row in the destination with_fivetran_deleted = FALSE. - A
DELETEin the source table updates the corresponding row in the destination with_fivetran_deleted = TRUE. - An
UPDATEin the source table updates the corresponding row in the destination with_fivetran_deleted = FALSE. - An
UPDATEof the primary key in the source table is treated like a new row being added. We add a new row in the destination with_fivetran_deleted = FALSE. The old row remains in the destination with_fivetran_deleted = FALSEuntil the corresponding table is re-synced.
History mode
- An
INSERTin the source table generates a new row in the destination with_fivetran_active = TRUE,_fivetran_start = <CURRENT_TIMESTAMP>, and_fivetran_end = 9999-12-31 23:59:59.999+00. - A
DELETEin the source table marks the corresponding row in the destination inactive with_fivetran_active = FALSEand_fivetran_end = <CURRENT_TIMESTAMP>. - An
UPDATEin the source table first marks the corresponding row in the destination inactive with_fivetran_active = FALSEand_fivetran_end = <CURRENT_TIMESTAMP>. We then add a new row in the destination with the same data and_fivetran_active = TRUE,_fivetran_start = <CURRENT_TIMESTAMP>, and_fivetran_end = 9999-12-31 23:59:59.999+00. - An
UPDATEof the primary key in the source table is treated like a new row being added. We add a new row in the destination with_fivetran_active = TRUE,_fivetran_start = <CURRENT_TIMESTAMP>, and_fivetran_end = 9999-12-31 23:59:59.999+00. The old active row remains in the destination with_fivetran_active = TRUE,_fivetran_start = <CURRENT_TIMESTAMP>, and_fivetran_end = 9999-12-31 23:59:59.999+00until the corresponding table is re-synced.
The history mode captures only the latest changes to a record at the time of sync. If a record is updated multiple times between syncs, we capture and sync only the most recent version.
Live mode
When using the No-trigger sync mechanism, column-store transparent tables are replicated in Live mode:
- An
INSERTin the source table generates a new row in the destination. - A
DELETEin the source table permanently removes the corresponding row from the destination (hard delete). - An
UPDATEin the source table is treated as aDELETEof the existing record followed by anINSERTof the updated record in the destination.
Live mode also applies to views and row-store transparent tables, regardless of the sync mechanism. For more information, see Replication of views and row-store tables.
Deleted columns
We do not delete columns from your destination.
Capturing deletes
Capturing deletes requires careful consideration because deletes can occur in data that was already imported in previous cycles and otherwise go unnoticed.
The method used to detect deletes depends on the Sync mechanism selected during connection setup. This setting cannot be changed after the initial Save & Test.
Default sync mechanism
NetWeaver Connection
To detect deletes, we use triggers and shadow tables created by the Fivetran NetWeaver API for each imported column-store transparent table:
- A shadow (
DELETES) table named/FIVETRAN/DELETES_<TABLE>(for example,/FIVETRAN/DELETES_T000). - A
DELETEtrigger on the source table that writes the deleted row’s primary key(s) into the correspondingDELETEStable.
During each sync, we read the DELETES table and apply the deletes in your destination.
Controlling where triggers are used
By default, Fivetran creates delete-capture triggers for all supported column-store transparent tables. To reduce overhead or gain finer control, you can limit deletes capture to specific tables in SAP using the Deletes detection using triggers parameter in transaction /N/FIVETRAN/CONFIG. When configured for selected tables, only those tables’ deletes are recorded and synced. Deletes on other tables are not captured. Inserts and updates continue to sync for all tables. For detailed steps, see Disable delete triggers for specific tables.
- If you manually alter triggers or shadow
DELETEStables, we cannot guarantee replication consistency. Doing so may require a full re-sync of all data. - If the structure of a source table changes (for example, after an SAP upgrade), the corresponding shadow table may become incompatible. In such cases, you must re-create the trigger and shadow table to avoid errors. For details, see Re-create shadow tables after source table schema changes.
For troubleshooting common issues with triggers and shadow DELETES tables in the NetWeaver Connection mode, see How to Troubleshoot Issues with Triggers and Shadow Tables in SAP ERP on HANA.
Shadow table maintenance
Available for Fivetran NetWeaver API version 1900271 and higher.
Fivetran automatically and safely maintains shadow tables to prevent them from growing indefinitely.
Our cleanup mechanism removes old records in batches once delete syncs are complete, keeping a 30-day safety buffer. It runs automatically in the background without affecting sync reliability or data accuracy and keeping the storage footprint low.
Shadow table maintenance is enabled automatically when your Fivetran NetWeaver API is version 1900271 or higher. No additional configuration is required.
HANA Database Connection
To detect deletes, Fivetran creates the following objects directly in your HANA database for each replicated table:
- A shadow table in the
FIVETRAN_DBschema named<SCHEMA_NAME>__<TABLE_NAME>, which captures the primary key values of deleted rows along with the deleting user (/FIVETRAN/USER) and timestamp (/FIVETRAN/TIMESTAMP). - An
AFTER DELETEtrigger on the source table that writes the deleted row’s primary keys into the shadow table.
During each sync, Fivetran reads the shadow table and applies the deletes in your destination.
If you manually alter the triggers or shadow tables in FIVETRAN_DB, Fivetran cannot guarantee replication consistency. Doing so may require a full re-sync of all data.
Shadow table maintenance
Fivetran automatically and safely maintains shadow tables in the FIVETRAN_DB schema to prevent them from growing indefinitely.
Our cleanup mechanism removes old records in batches once delete syncs are complete, keeping a 30-day safety buffer. It runs automatically in the background without affecting sync reliability or data accuracy.
Shadow table maintenance is enabled automatically. No additional configuration is required.
No-trigger sync mechanism
This sync mechanism is available for the NetWeaver Connection mode only. The HANA Database Connection mode does not support the No-trigger sync mechanism.
Fivetran detects deletes by maintaining internal snapshots of existing rows for each table. During each sync, we compare the current state of each row against the stored snapshot to detect inserts, updates, and deletes:
- Rows present in the source but not in the snapshot are treated as new inserts.
- Rows present in the snapshot but no longer in the source are treated as deletes and permanently removed from the destination (hard deletes).
- An update in the source is treated as a delete followed by an insert in the destination.
The No-trigger mechanism applies only to column-store transparent tables. This does not affect views or row-store transparent tables. For more information, see Replication of views and row-store tables.
Re-sync partition and re-sync table scenarios
The DROP and TRUNCATE operations have certain side effects that can impact the reliability of your connection. Fivetran can detect and automate specific processes without disrupting the data replication:
- In the case of a
TRUNCATE(partition or table) operation: Fivetran automatically detects truncations for each partition and initiates a re-sync during the next sync cycle. - In the case of a
DROPpartition operation (typically done viaALTER TABLE): Fivetran does not support automatic detection of this operation. Therefore, you must manually re-sync your table if one partition is dropped. - In the case of a
DROP TABLEoperation: No further action is taken for that table, and it is treated as if there are no updates. The data remains intact on the destination.
Create partitions
Creating new partitions for any table is automatically detected by Fivetran. These are synced normally during the next sync cycle.
Fivetran also detects modified/restructured partitions, triggering a re-sync for those partitions.
Log replay delay tolerance during failover
In high-availability SAP HANA replication setups, a failover from the primary to the secondary instance may involve a short log replay delay.
To ensure data consistency without unnecessary re-sync, we support a log replay delay of up to 10 minutes between replicas during failover. If the delay is within this threshold, tables continue syncing without re-sync. If the delay exceeds 10 minutes, affected tables are re-synced individually based on the detected delay.
Unpack data
This section applies to the NetWeaver Connection mode only.
We support unpacking for two tables, STXL and MDTC. During unpacking, we may remove certain columns and generate others. The generated columns are not included in the list of filterable columns when applying filters to the table. For more information, see Filtering rows based on column values. Because of the unpacking process, the record counts in the source and target may not match.
STXL unpacking
This feature is available for the Fivetran NetWeaver API version 1000202 and higher.
We unpack the long-text STXL table into a readable and transparent format. We have modified the table structure to account for this. The following is the replicated STXL version:
Original STXL | Replicated STXL |
|---|---|
| MANDT (PK) | MANDT (PK) |
| RELID (PK) | RELID (PK) |
| TDOBJECT (PK) | TDOBJECT (PK) |
| TDNAME (PK) | TDNAME (PK) |
| TDID (PK) | TDID (PK) |
| TDSPRAS (PK) | TDSPRAS (PK) |
| SRTF2 (PK) CLUSTR CLUSTD | /FIVETRAN/STXL_TEXT |
We replaced the SRTF2, CLUSTR, and CLUSTD columns with the new /FIVETRAN/STXL_TEXT column. The aggregation of these three columns contains the necessary information to decode the human-readable text for the new column. The record count in the source will be greater than that in the target.
MDTC unpacking
This feature is available for the Fivetran NetWeaver API version 1900241 and higher.
We unpack the MDTC table into a readable and transparent format. We have modified the table structure to account for this. The following is the replicated MDTC version:
Original MDTC | Replicated MDTC |
|---|---|
| MANDT (PK) | MANDT (PK) |
| RELID (PK) | RELID (PK) |
| SRTFD (PK) | SRTFD (PK) |
| MATNR | MATNR |
| WERKS | WERKS |
| CLUSTR CLUSTD | DTNUM (PK) DTPOS (PK) * (other non-PK unpacked columns) |
We have replaced the CLUSTR and CLUSTD columns by adding 59 new columns:
- We expanded the primary key with two columns,
DTNUM (PK)andDTPOS (PK). - We added 57 non-primary key columns containing unpacked data.
Unpacking MDTC is a many-to-many type of unpacking — one or more records from the source could be unpacked into one or more records in your destination, in human-readable format.
The MDTC table replicates data in Live mode only, so deletes that occur in your source become hard deletes in your destination. The record count in the source will be less than or equal to that in the target. SAP Archiving is not supported for this table.
Handling deleted data during SAP archiving process
This feature is only available on NetWeaver Connection mode and for the Fivetran NetWeaver API version 1900216 and higher.
You can mark records archived by an SAP archiving process as non-deleted in your destination by checking the Start tracking archive deletes box under SAP Archiving job users in the Fivetran's SAP configuration screen to enable this feature.
Once checked, this setting cannot be undone. This change is applied at a given SAP source. This means that all connections replicating data from the same SAP source will have this feature enabled.
When this setting is enabled, an SAP Archiving job users input field appears. Enter the username(s) under which the SAP Archiving job runs.
When you have enabled marking archived records as non-deleted for the SAP source, Fivetran does the following:
- A new column is added to all tables synced from the relevant source in your destination,
_fivetran_sap_archived(BOOLEAN). The column value for the previously synced data becomesNULL. - The
_fivetran_sap_archivedcolumn value is set toFALSEfor the records synced after the feature was enabled. - The
_fivetran_sap_archivedcolumn value is set toTRUEfor the records archived on the SAP side if the user committing the delete is included in the input list of users. The specified username and the username of the SAP user related to the SAP archiving process must be an exact match. In all other cases (for example, if the box was checked, but the input user list remains empty), the_fivetran_sap_archivedcolumn value remainsFALSE. - If this column is
TRUE, then the corresponding record will not be marked as deleted. For example, depending on the sync mode, such record continues to have_fivetran_deleted=FALSEin case of soft deletes or_fivetran_active=TRUEin case of history mode.
See our Configure Fivetran in SAP documentation for more information about configuring SAP archiving deletes.
The handling of the SAP archiving process is not supported for the following:
- Unpacked
MDTCtable. For more information, see the MDTC unpacking section. - Views and row-store tables. For more information, see the Handling deletes section.
Monthly Active Rows calculation
The records archived by SAP archiving processes are treated by Fivetran as either updates (if this feature is enabled) or deletes (if the feature is disabled) and count towards paid MAR. The standard rules for Monthly Active Rows (MAR) still apply for these operations.
Replication of views and row-store transparent tables
This feature is supported in both connection modes. For the NetWeaver Connection mode, it requires Fivetran NetWeaver API version 1900216 or higher.
Fivetran allows you to replicate data from:
- Row-store transparent tables
- Column-store transparent tables
- Views
Due to structural limitations in views and row-store transparent tables, we cannot incrementally sync them. Instead, we perform a full sync of these objects at each sync cycle. Fivetran handles records as follows during the sync of views and row-store transparent tables:
- If a record doesn't exist in the destination, we insert it into the destination.
- If a record exists in the destination, we update the existing record.
- If a record exists in the destination but is not present in the latest sync cycle, we permanently delete it from the destination. Views and row-store tables replicate data in Live mode only.
Handling deletes
Since views and row-store transparent tables are fully re-synced at each cycle, Fivetran does not use the delete-capturing mechanism. As a result:
- We do not create triggers or shadow
DELETEStables - We do not add the Fivetran-generated columns for tracking deletes:
_fivetran_deleted(soft delete mode)_fivetran_active,_fivetran_start, and_fivetran_end(history mode)_fivetran_sap_archived(archive deletes).
Primary keys and unique identifiers
Views don't have primary keys. To ensure uniqueness, we introduce the _fivetran_id (LONG) column, which auto-increments for each record.
In rare cases, a view may appear to have a primary key if it is based on a single underlying table. When this happens, the view inherits the table’s primary key. However, Fivetran automatically discards these primary key constraints.
Monthly Active Rows calculation
The standard rules for Monthly Active Rows (MAR) still apply to views and row-store transparent tables.
MANDT filtering
This feature is only available on NetWeaver Connection mode and for the Fivetran NetWeaver API version 1900230 and higher.
When logging into an SAP system, the login is client-specific, and data is automatically filtered by client (MANDT). Our connector has broader data access and can replicate data for all clients, regardless of the logon client (client-independent data selection). You can enable MANDT filtering in the Fivetran's SAP configuration screen to limit the replicated data to only data relevant to the connected client.
For more information about enabling the MANDT filtering, see our Configure Fivetran in SAP documentation.
Excluded objects
Fivetran does not sync the following objects:
- Database-level system tables
- Tables and views not present in the SAP ABAP Dictionary (NetWeaver Connection only)
- Temporary tables
- SAP functions
- Projection views
- Parameterized views
- Tables with underlying proxy objects (for example, compatibility views). However, it is possible to replicate compatibility views themselves, as they are treated as views. For more information, see Replication of views and row-store tables. Example:
- In some SAP systems, table
MBEWis based on compatibility viewMBVMBEW. Do not replicate tableMBEW, as it may be empty, contain partial information, or, in rare cases, include unexpected data. Instead, replicate the viewMBVMBEW.
- In some SAP systems, table