Events
Event tracking measures consumer behavior on a website, app or emails, usually through pixel tracking. Fivetran's integrates with several services that collect events sent from your website, mobile app, or server and then loads these events into your destination.
Supported services
Fivetran offers support for the event tracking libraries:
- Amazon Kinesis
- Snowplow analytics (open source)
- Segment
- Webhooks
See our Swagger documentation for more information about webhook endpoints.
Sync overview
After instrumenting the tracking code on your website, server, or mobile application, Fivetran's event pipeline will collect, enrich, and load all of this data into your destination in near real-time. In addition to being easy to set up, Fivetran is built to scale to hundreds of millions of events per day, and automatically retain a secure backup of your event data. If your destination is ever compromised, we can easily reload all your events for you.
The following diagram outlines the process for event collection at Fivetran:
When we receive the events in our collection service, we first buffer the events in the queue, then store them in our Google Cloud Storage (GCS) before writing the events to a temporary file. We push the events (the temporary file) to the destination when the sync runs.
NOTE: We do not sync the events that are in our queue or are getting stored in GCS while a sync is in progress. We process these events during the next sync.
Updating data
Fivetran takes the new data and appends it to the existing tables. When we encounter events that cannot be parsed (such as incomplete data) we skip those events and alert the user.
Fivetran is always collecting data and stores it long term in a staging S3 bucket. The changes are then loaded on a 15 minute interval by default. This upload time can be changed inside of your Dashboard.
Fivetran does not propagate deleted data for events, because the events have already happened.
Schema changes
Fivetran does not propagate schema changes. If you stop tracking a metric, it metric will have a null value for that row. All the historical data would still exist. If you stop tracking a table, the table will still exist, but you won't get any new rows. It's not possible to delete a row in the source, because the event has already occurred.
Webhooks connection mapping
Fivetran supports a webhook integration for POST-ing arbitrary data directly into your destination. Webhook connections occur at the table level. Each different connection that is created within Fivetran (shown below in the blue connection icon circle) is given a specific URL to post data to. Any data posted to that URL will be added to a single destination table.
Other event tracking services connection mapping
Fivetran supports a few pre-existing tracking services, namely Segment and Snowplow. Each different connection that is created within Fivetran is given a specific URL to post data to (shown below in the blue connection icon circle). You can use either Segment or Snowplow tracking services and send all of the events to your a specific Fivetran URL. Differing from the generic webhooks connection, any data posted to that URL will be automatically normalized into a set of destination tables within the same schema.
Naming
For Webhooks, you can name destination table in the Dashboard while creating the connector. Fivetran will create the new destination table for you automatically, and it's best not to have that table already created. You can also designate whichever schema you would like the table to reside in. If the schema that you select does not already exist, Fivetran will automatically create it for you in your destination.
For Segment or Snowplow, you can name the destination schema in the Dashboard while creating the connector. Within your destination schema, we will create a set of default tables for you in your destination.