The secret sauce behind Databricks’ marketing analytics growth

The Situation

Meet Chris Klaczynski: a Marketing Analytics Manager and Fivetran power user at Databricks. As Databricks’ first full-time Marketing Analytics team member, Chris brought a wealth of experience from Amazon’s Kindle and Alexa Customer Behavior teams when he joined in 2019.

Chris’ role was clear: to help marketers at Databricks solve their toughest problems. Databricks – a business that in turn prides itself on helping data teams solve the world’s biggest challenges – was founded in 2013 by the original creators of Apache Spark, Delta Lake and MLflow. Built on a modern Lakehouse architecture in the cloud, Databricks combines the best of data warehouses and data lakes to offer an open and unified platform for data and AI.

The Problem  

Chris’ first responsibility at Databricks was supporting the primary marketing objectives: driving pipeline generation, growing the database, and improving ROI. But as Databricks rapidly expanded, the need for centralized and documented data became more and more apparent. Data silos were appearing around the business, including on Chris’ marketing team, where data was stored in its own data warehouse.

It was critical that Chris’ newly-built dashboards were supplied with trustworthy, timely data for marketing operations to keep running smoothly – but without dedicated engineering resources, and in the face of a rapidly expanding marketing team, scaling with demand became next to impossible.

“97% of what I was doing was keeping the lights on. Getting out of the data hell was a big quality of life improvement for me.”

When Chris joined Databricks, data was cobbled together through Notebooks, Alteryx and legacy pipelines from central teams. It wasn’t a scalable approach, and Chris would deal with slews of emails from Marketing leaders who didn’t trust their data and reports. Databricks faced a number of challenges with their traditional data warehouse:

• Issues with their Salesforce and Marketo pipelines in particular – they weren’t able to get data into the Delta format natively, and needed to use Parquet as a workaround.
• Issues appending data natively to existing tables required using temporary tables and more convoluted ETL.  
• Schema changes were always breaking pipelines, resulting in outages and stale, untrustworthy data.

Because of this, stakeholders weren’t able to discern if a juicy insight was real, or simply a data quality issue. Chris’ team lost trust from stakeholders every time their data didn’t line up with source systems. This hampered planning, budget-allocation and other decision making.

[CTA_MODULE]

The Solution

Chris began looking for a low-code, turnkey solution that would give his team the reliable pipelines they needed. A three-point strategy informed his search:  

• Get out of the data engineering ‘jungle’ and into insights and predictive analytics.
• Hire analysts and Digital SMEs, not Data Engineers and DevOps.
• Be self-sufficient with our data pipelines and utilize Delta and AutoML.

“We didn’t want to be in the data engineering business – we wanted to focus on delivering insights and running analyses that generate business outcomes.”

Fivetran immediately stood out as a solution to Chris’ data pipeline needs. In trialing the product, the setup was quick and simple. “We followed the quick instructions, set up a cluster, whitelisted the IPs and we were off to the races within a few days.”

Chris’s team now uses Fivetran to bring in data from all of their core marketing source systems: Marketo, Salesforce, Facebook Ads and Google Analytics. “Simply put, it just works.” Once in their lakehouse, Chris’ team joins the data with other sources from Product and Central Teams, transforming it in a meaningful way to run Data Science, machine learning, and generate vital Tableau dashboards for analysis.

The Outcome

Chris now supports a rapidly growing data team of analysts and leaders who, in turn, deliver crucial insights to over 90 marketers. In total, Fivetran saves Databricks over 40 hours a month in engineering time – the equivalent of a part time Data Engineer.

“Instead of hiring the kind of traditional engineering headcount, Fivetran allows us to focus on business value, hiring analysts, dashboard builders, people who are experts in web analytics and paid media. Our infrastructure is a lot broader and more advanced than it was a year or two ago.”

By bringing on Fivetran to handle data ingestion, Chris freed up time to build a full suite of Tableau dashboards that answer the most common questions that his marketing team face, saving his team time and allowing them to focus on adding value with three Data Science and machine learning projects:

• Accurate campaign-level forecasts to allow managers to understand how they’re pacing and whether their performance is on-pace.
• A causal impact model which determines the impact of a specific customer event on their consumption.
• A propensity model that predicts event attendance based on previous behavior, allowing teams to be more targeted in their email marketing.

The downstream impacts of Chris’ decision making are already being felt internally at Databricks. Customer Service and Customer Success have seen how easy the process has been and are asking for more use cases and connectors to be switched on.

“By using Fivetran, we can bring in new sources of data that our new hires will be using  – areas we haven’t explored deeply. They can hit the ground running and start building dashboards immediately. They don't have to build pipelines or get familiar with notebooks or writing code. They can have immediate access to data so they can focus on insights and building relationships with their stakeholders.”

Access to reliable data has been critical for marketing account managers, who are now more accountable for the forecasts they are making. Increased visibility through sophisticated dashboards has made it easier for teams to track and rapidly act on campaign performance.

Ingesting data into a Delta Lake? Check out the Databricks Partner Portal to learn about the power of Databricks and Fivetran.

Visit Fivetran’s Marketing Analytics page to see how you could be centralizing your marketing data in a way that’s fast, reliable, and in real time.

[CTA_MODULE]