On the heels of the Modern Data Stack Conference 2021 and our merger with HVR, I wanted to take a moment to talk about what’s next for our incredibly fast-growing company.
Everything we develop at Fivetran simplifies data access for our customers — our mission is to make that process as simple and reliable as electricity. That vision will continue to guide us into 2022 and beyond.
Going forward, we’ll remain obsessively focused on four areas: connector reliability, data security, data transformation and powering customer data products. Those last two areas are key because our customers need to model their data for analytics, and many need to turn it into analytics products for their own customers.
Here’s a look at how we’ll be accelerating our progress in each area.
This post is adapted from a recent talk I gave at the Modern Data Stack Conference 2021, along with Mark Van de Wiel, CTO at HVR. Our talk is available on demand.
Pushing beyond 99.9 percent reliability
Our highest priority will continue to be reliability. Over the last year, reliability has increased from 99.75 percent to nearly 99.9 percent — and that includes third-party sources and destination errors that are beyond our control.
In the same time period, the number of pipelines that we manage doubled, to well over 70,000. At any given time, fewer than 100 pipelines are broken, and we’re committed to reducing that number going forward. Already, we’re saving data engineers and data analysts hundreds of thousands of hours in pipeline maintenance and repair work — hours that data teams can reinvest in product refinement and data analysis.
For the foreseeable future, reliability will remain the highest priority for hundreds of Fivetran engineers.
Protecting enterprise data
Fivetran Business Critical is another incredibly important pillar of our offering going forward. Available now for AWS customers, with availability for Microsoft Azure in private preview, Fivetran Business Critical is a powerful feature set for the highest throughput and highest levels of security, enabling direct and secure connectivity between your VPC, Fivetran and your data warehouse. All of them are connected by a private link to avoid the public internet — and all this occurs within your AWS or Azure region.
Using high throughput connections within the data center, all of the data being moved, both in transit and at rest, is encrypted with your keys. And if you need PCI compliance, which is for finance companies that deal with credit card data, we can do that for you, too.
Looking ahead, we're adding U.S.-only support for enterprises and other businesses with the strictest security and compliance requirements.
Integrating data transformations and modeling
Another key focus area for us: integrated data transformations and data modeling. I’m thinking of one customer in particular — a cloud-based financial planning solution that helps their clients analyze financial data from sources like NetSuite and Oracle. Every time Fivetran syncs that data into a data warehouse, transformations need to run, to materialize the data models for their downstream clients to query. Data freshness here is key.
This will sound familiar to many of our customers. They need up-to-date dashboards with the latest data, but they don't want to run expensive transformations unnecessarily.
In private preview, we now offer integrated scheduling for dbt transformations — your connectors and transformations all scheduled together to minimize latency between data arriving and dashboards being up to date. The response thus far has been phenomenal; we can't wait to roll it out to you in the coming months.
Speaking of dbt, we offer more prebuilt dbt data modeling packages than any other provider. These packages provide clear, comprehensible tables to power your reports, visualizations and dashboards — and they’re extensible, so you can deepen your analysis as necessary.
This kind of integrated experience is the future of Fivetran.
Powering analytics products
Powered by Fivetran (PBF) is our solution for organizations that need to onboard customer data from external sources, and it will be a critical part of Fivetran going forward. One important PBF customer is marketing analytics platform CaliberMind. The CaliberMind application combines customers’ ad, sales and marketing data, analyzes it, and delivers essential metrics and insights.
To support the app initially, CaliberMind had to build and maintain an entire custom data pipeline solution, which consumed a huge amount of data engineering resources. Now it uses Powered by Fivetran to onboard customer data — via its own branded UI — and, as CEO Eric Westerkamp puts it, “That engineering time can be spent on our core intellectual property: creating valuable insights for our customers.”
Here’s a look at how CaliberMind customers use Fivetran within the CaliberMind UI:
Fivetran and the Modern Data Stack Conference
As mentioned above, this post is adapted from a talk on the future of Fivetran that I delivered at the Modern Data Stack Conference 2021. Head to our resources center to watch the full talk, or check out any of the 40+ keynotes and sessions. The conference covers pretty much every topic data professionals care about: data analytics, data engineering, product development, data security — and many more.