Fivetran and the Hierarchy of Data Needs
In recent years, a number of data professionals have independently arrived at a hierarchical model of data-related business needs. Like Maslow’s famous hierarchy of psychological and emotional well-being, the needs are organized from the most basic to the most rarefied, with higher needs essentially dependant on lower ones.
Image by Monica Rogati
While there are several versions of this model, they all have major concepts in common. The models move from the the collection of raw data at the base, to the storage and management of data, to everything related to exploratory analysis and simple analytics, to testing and modeling, and, at the very top, artificial intelligence and deep learning.
The topmost activities are extremely difficult to perform without a solid foundation in data collection and management. As a result, highly qualified data professionals who are eager to test models or delve into machine learning find themselves collecting and managing data instead. It is well-known that 80% of the average data scientist’s time is consumed by such activities. Many data scientists, especially at smaller or short-staffed organizations, are shanghaied into data engineering work for which they may have no particular training (or aptitude).
Fivetran Helps You Move Up the Hierarchy
At Fivetran, our goal is to take care of the basic needs at the bottom layers of the hierarchy: the collection of raw data, and its storage and management. Starting with our connectors, we automate and fully manage the process from the collection to the storage and management of the data. We are advocates for the flexibility of ELT, which consists of the large-scale collection and centralization of data prior to its transformation. Our approach saves thousands of hours of manual reporting, building, and maintenance work for our customers and eliminates the need for our customers’ analysts to predict every quantitative business problem that might need to be solved.
Although we are not a business intelligence or warehousing company, our analyst recipes are intended to give you just a bit of a gentle push beyond the basic needs of the hierarchy. We want you to direct your attention to exploring and analyzing your data, not wrangling with it. Longer-term, we hope you will be able to shift your attention to the pinnacles of modern data usage — machine learning and artificial intelligence.
Access to data and basic data literacy are essential starting points for any organization that hopes to contend in its industry. Get in touch with us, and we will speed you well along your way to making the fullest use of your data, instead of fighting with it.