Snowplow

Snowplow Analytics is an event tracking platform that allows you to track events across all channels and action the data in real-time. Snowplow uses trackers to track your information in whichever language your project is built.

Features

Supported trackers

We pull information about your events from the code that tracks your events. We officially support tracking in JavaScript, but can integrate tracking events from other languages such as Ruby and Python.

Enrichments

We support the following Snowplow enrichments:

NOTE: Your JSON configuration file must follow the Snowplow schema format. You must use the enrichment-specific name (see the Notes section of the table above) for your configuration file.

To disable an enrichment, in the JSON configuration file, set the enabled field to false.

Setup guide

Follow our step-by-step Snowplow setup guide to connect Snowplow with your destination using Fivetran connectors.

Sync overview

After instrumenting the Snowplow tracking code on your website, server, or mobile application, Fivetran's robust event pipeline will collect, enrich, and load all of this data into your destination in near real-time:

1. Snowplow events sent to webhooks.fivetran.com

2. Events collected

3. Events queued

4. Events processed and enriched

5. Events prepared (transformed and cleaned) and written to CSV file

6. Loaded into your destination

Snowplow does not allow Fivetran to sync your historical data. We can only capture data from your connection date forward.

Connection mapping

Each connection that is created within Fivetran (illustrated below in the connection icon circle) is given a specific URL to post data to. You can use Snowplow tracking services and send all of the events to the specific Fivetran URL. Any data posted to that URL is automatically normalized into a set of destination tables within the same schema.

Naming

You can name the destination schema in the Fivetran dashboard while creating the connector. Within your destination schema, we create a set of default tables for you in your destination.

Schema information

Snowplow has standard and custom fields. Each integration creates a single schema, shown below. If you choose to use custom events, each custom event type will have its own table.

EVENT

Snowplow has a primary standard table, EVENT. Fivetran deduplicates events based on the event_id and collector_tstamp columns. If two events have the same event_id and collector_tstamp, we overwrite the first event.

The columns in the EVENT table have the following data types:

UA_PARSER_CONTEXT

Fivetran follows Snowplow's UA parser enrichment to parse User agents into a separate context.

The user agent parser contexts (UA_PARSER_CONTEXT table) have the following columns:

BAD_EVENT

If Fivetran detects Snowplow events that don't conform to the Snowplow Canonical Event format, we add the BAD_EVENT table to your destination schema to store these events. The columns in the BAD_EVENT table have the following data types:

Load custom fields

If you implement Snowplow's custom click feature, all the custom clicks will be loaded into the EVENT table in your destination's Snowplow schema.

Fivetran supports unstructured events. You can track your custom objects.

To load custom fields into your destination, do the following:

1. Write a JSON schema describing your custom attributes known as an iglu file in the Snowplow documentation. Iglu files are JSON schemas and Snowplow's integration is setup to work with JSON schemas. For more information you can see an explanation of how JSON schemas work.

2. Validate your schema before you start sending your data to us.

3. We don't have a repository that's analogous to Snowplow's "iglu", so just put your schema at a regular URL such as a public github repository, such as: https://github.com/fivetran/snowplow-schemas/blob/master/hello_world.json or https://github.com/fivetran/snowplow-schemas/blob/master/test_table.json

4. Use the URL that you established in step 3 as the schema URL when you call Snowplow:

   window[snowplowName]('trackPageView', null, [{
   schema: 'https://raw.githubusercontent.com/fivetran/snowplow-schemas/master/hello_world.json',
   data: {
   hello: 'Hello world!',
   hello_array: ['Hello', 'world!']
   }
   }]);
   

5. Make sure to have a "title" attribute for the schema, which will become the name of the table where the data gets stored. For example, the following will create a table named example (id int):

   {
   "title": "example",
   "properties": {
   "id": { 
       "type": "integer"
       }
   }
   }
   

6. If you want to create a decimal type (for example, for storing currency values), just use the multipleOf property:

   {
   "title": "example",
   "properties": {
       "value": { 
       "type": "number",
       "multipleOf": 0.01
       }
   }
   }
   

   This will create a destination table example (value decimal (18, 2)).

7. If you want to store an enum value, add a type associated with it. All the enum values should be of same data type:

   {
   "properties": {
       "value": { 
       "type": "string",
       "enum": ["value1", "value2"]
       }
   }
   }
   

NOTE: We only support values and homogenous arrays. Nested objects and tuple-style arrays are stored in but not extracted into your destination since they don't have an obvious representation.

Exclude source data

We don't yet have a way to exclude source data. You can independently, however, limit the data you send to Fivetran.

Sync limitations

We only support publicly-hosted schemas in Iglu or GitHub.