Honeycomb builds sweet and scalable data foundation

‍Honeycomb, the pioneers of the term “observability” for next generation production system monitoring, enables companies to improve their software ownership so that their engineering teams can continue to run more efficient processes and improve customer satisfaction. Companies in all industries, such as Discord, Mode Analytics, LaunchDarkly, Optimizely, Intercom, and more rely on Honeycomb’s technology to improve their customer experiences.

When Irving Popovetsky, Head of Customer Engineering at Honeycomb, recognized that there was a data analytics visibility issue interfering with his quest to understand the customer journey, he also realized that someone needed to step up and build a scalable data analytics stack. The realization came to a head during their recent exercise to make pricing more customer-friendly.

The Challenge: Data Analytics Through Manual CSV

As most companies do in earlier data maturity stages, Honeycomb piped product events to MySQL RDS and directly queried them through Metabase. Through these queries, they were able to understand how customers were using their product, but were unable to directly connect usage with consumption and revenue. This led to multiple, frequent CSV extracts from a combination of production database data from Metabase as well as their financial reporting systems and CRM data. This initial project took over 3 months to complete and prompted Irving to begin his search for tools to establish a scalable data stack that the entire organization could easily leverage.

Building A Data Stack From Scratch

In addition to the systems mentioned above, other departments also had tools that hosted useful data but weren’t integrated into any centralized database. Popovetsky explains:

We could have something like one to two ETL developers who are just sitting there and operating the ETL, but we don’t have to think about the operations of the extract and load pipeline which definitely takes a ton of work off our plates.

With this in mind, Irving reached out to multiple consultants for expert advice on implementing a new data stack for Honeycomb. Ultimately, Irving chose Modern Data for their comprehensive experience, broad customer base, and history of positive results.

After choosing Mode Analytics for their flexibility and extensibility and Snowflake for their powerful storage and query capabilities, all that remained was choosing a data integration provider.

Why Fivetran?

Honeycomb knew that a large amount of data was spread across multiple applications and a production database, and that they needed an easy way to centralize and cross-reference all of this data to uncover better insights into the business. Irving had access to limited engineering resources, so he knew that he had to minimize maintenance for the broken data integrations that commonly occur with frequent database schema changes and API changes.

Irving immediately saw benefits after his first integration, but continued to do his due diligence in evaluating Fivetran:

With other tools we would’ve had to design the schema and think about what the target schema looked like. Fivetran made all of the point-and-click easy and we can use the target schemas out of the box.

Through his tests, he also found:

• Source and destination support. Fivetran has pre-built connectors for both the immediate sources as well as future projects.
• Security. Fivetran offers a variety of methods to securely connect to production databases to ensure that all replicated data is safe.
• Well designed schemas. Most applications come with a standardized schema diagram (ex: Stripe) that shows the entire breadth of data extracted, as well as table relations, shortening time to insight when building reports and dashboards.
• No maintenance. Connectors made by Fivetran automatically update data as read from the source, and don’t require any manual interference for any sources, saving time for insight generation.

During his evaluation, data sources were liable to change, and usage could ramp up and down as the business continued to configure its data sources. This made projected short-term usage of Fivetran more volatile and ultimately influenced the decision to use a credit card for monthly billing and more flexibility in usage. After finalizing data sources and determining which would be used in future projects, Irving got in contact with the Fivetran sales team, which was able to dedicate technical resources for best practices and walk through different packages.

Being able to spend a couple of months using a credit card while we figured out our long term usage was very helpful. From there, we were able to easily move on to an enterprise plan. It felt very low-risk to keep adding connectors and progressing.

Reporting for Finance, Sales and More

The first project that Honeycomb tackled was a financial analytics project to determine MRR (monthly recurring revenue) and ARR (annual recurring revenue) from data spread across Stripe and Salesforce to share company health metrics to stakeholders.

The team at Honeycomb now has their sights set on improving Customer Health scores and user analytics reporting:

• Customer health scores are calculated from user engagement, tracked in systems such as Intercom, and are used to identify the best customer expansion opportunities or potential customer churn risks.
• User analytics have the goal of identifying prospective customers to proactively engage in sales cycles and they surface these customers through embedded Mode directly into Salesforce.

Data analytics at Honeycomb used to require SQL knowledge and multiple weeks of stitching together disparate spreadsheets, but has now become so accessible that individual sales representatives are customizing their own dashboards and reports with ease. Irving shares that while the greater organization may not know what Fivetran does, they know that they're benefiting from the end result:

Very rarely do we have to tell anyone anything other than: Fivetran exists and it’s magically making all of the data appear in Snowflake.

[CTA_MODULE]

About Fivetran: Shaped by the real-world needs of data analysts, Fivetran technology is the smartest, fastest way to replicate your applications, databases, events and files into a high-performance cloud warehouse. Fivetran connectors deploy in minutes, require zero maintenance, and automatically adjust to source changes — so your data team can stop worrying about engineering and focus on driving insights.

About Snowflake: Snowflake delivers the Data Cloud — a global network where thousands of organizations mobilize data with near-unlimited scale, concurrency, and performance. Inside the Data Cloud, organizations unite their siloed data, easily discover and securely share governed data, and execute diverse analytic workloads. Join the Data Cloud. https://www.snowflake.com/

About Mode: Mode is a data analysis platform that combines a powerful, web-based SQL editor with charting and sharing tools.