Fivetran partner Archetype Consulting discusses cloud migration, from initial considerations to long-term benefits.
In the following interview, Max Gomez, Co-Founder & Analytics Practice Lead for Archetype Consulting, offers insight into cloud migration, including its importance, business concerns, and short and long-term benefits.
Whether they’re rolling out new ERPs, CRM systems, or point-based applications they need to support the business, cloud-first businesses are moving away from purchasing hardware or solutions that require internal resources to build and maintain. They’re looking to the cloud for solutions that offer speed and flexibility and the ability to scale up and down based on resource needs. Through cloud applications, businesses can get the efficiencies and economies of scale while the provider makes sure everything is working from a technical standpoint, saving internal resources.
Security is probably the top one. Businesses can overcome security concerns by working with cloud solution providers and having a discussion around their concerns. You should do an internal test of the products, have an employee try to poke holes at the architecture and strategy to figure out if there are ways that the environment can be penetrated. Make sure that you know the measures that the provider has taken to make its product as secure as possible. But the flip-side is that there are equally as many, if not more, security concerns for on-prem. Breaches can occur without cloud providers. Security is always a risk but it’s about due diligence and what measures you’re taking to prevent the breaches.
Cost is another concern that comes up. We often talk clients through the full modern data stack — tools like Fivetran, that get the data from your source system in near-real-time into a cloud data platform like Snowflake, and a BI visualization tool such as Tableau, Looker or Sigma. They’re concerned about the total cost and that once they start adding data sets in the future, the pricing models will change. But the cost isn’t constantly adding up, the tools aren’t running all of the time, only when you’re computing or materializing the data. We can help them control the costs and estimate what the rough numbers will look like. In these cases we have to show people that there are other benefits here that far surpass that initial cost.
Scalability. This is particularly important for companies that are dealing with larger volumes of data. You have the resources available to access your growing data when you need it, to increase your virtual warehouse sizes to handle big queries and scale down when queries complete.
Ease of use. Cloud solutions are being built with the mindset that we’re in a new age where everything needs to be easy to use. Generally, you don’t need to use your technical resources for setting up the architecture.
Cost Savings. Since you won’t have to spend on physical equipment, utilities, maintenance, and other pertinent expenses, your operating costs will be significantly minimized. Also, when your business resources are on the cloud, if there is a need for extra hardware capacity, resources can be easily spun up as needed without having to procure new hardware that would sit idle during times of less demand.
Instant Access. The cloud makes it easier for you to access and retrieve data, even in the middle of the night or when you’re traveling. This is because the cloud never switches off; it is always at work and all you have to do is find a reliable internet connection. There is less downtime, so your users can access the data.
I’m willing to bet that nearly 100% of companies are in the cloud in some capacity, like in Office 365 or Google Drive. There is probably someone in the business leveraging the cloud for business and cost-efficiency. In terms of analytics and modern architecture, I honestly cannot think of a great reason for a company to say no to the cloud.
People need to decide what cloud provider they’re using — the big ones being Amazon, Microsoft and Google. Next you need to figure out your data sources. You may have sources that already exist in the cloud, like Salesforce or Hubspot, in addition to on-prem legacy systems.
Determine where you want your source data to be collected and stored. Snowflake is the leading cloud data platform so that is generally the target that customers want to get their data in. Then you need a data integration tool like Fivetran to replicate that data from the source system into your warehouse. A BI tool, like Looker or Tableau, to visualize and make use of the data is the final step.
To realize the full potential of the cloud beyond the clear technical benefits, you need to start by answering a few questions to understand your business case and what you’re trying to accomplish. What is the problem you’re solving? What is the solution you need? Who is going to benefit from a user standpoint? What is the rough current cost versus future costs and estimated savings over 1,3, 5 years or longer?
Once you have those questions answered, you need to understand the dependencies. Do you need external support and expertise to get this done? Who are the internal stakeholders? Where does the decision-making capacity live? Do you have the executive sponsorship to back the project and strategy?
Once you have these things lined up you can better decide what tools you’re going to use. Doing a proof of concept or iterative output process will help prove the value to sponsors and consumers.
Resource constraints. The same resources supporting the current environment or other initiatives may also be tasked with learning new technologies and getting them implemented. Some businesses need support for best practices when it comes to a new architecture. For instance, we may do a small hands-on session with a client to get them the information they need and to ensure efficiencies and the correct approach.
The speed to implement is an immediate benefit. You can access new datasets and generate analysis or insights within a shorter time frame than ever before.
The cost benefit, although often an initial concern, is a long-term benefit. It can be immediate, but extrapolating that out over five years will likely be an impressive number. There are also cultural impacts, such as training and education with a data-first mentality. The more of these easy-to-use data platforms you have, the more attractive you’re going to be to data professionals who are looking for jobs with businesses that take data and modern technology seriously. They want to spend more time doing analysis or data science rather than wasting time manually cobbling data together.
We worked with ASA in Boston at a point when they were transforming their business model. Previously, ASA targeted high school, college and mature students to provide financial aid services. The mission has since changed to help with career development and job market education for students ranging from middle school through college. As part of this shift, there was a considerable increase in the quantity and content of source data and developing an analytics environment strategy was critical.
Our initial project focused on identifying data integration, storage, and visualization tools that ASA could maintain with minimal IT resources. Over a six-week period, we created a near-term and long-term data strategy including a tool proposal, implementation roadmap and an enhanced data governance model.
The second phase of our project with ASA focused on the execution of the proposed roadmap: installing Fivetran to integrate existing and new data sources, Snowflake as a cloud data warehouse, and Tableau for data analysis and visualization. With this modern data environment, ASA can shorten the time to develop key insights and measure progress towards their business goals. The project has completed on-time and on-budget.
About Archetype Consulting: Archetype Consulting is a management and IT consulting firm, supporting clients through data analytics and business solutions.
About Fivetran: Shaped by the real-world needs of data analysts, Fivetran technology is the smartest, fastest way to replicate your applications, databases, events and files into a high-performance cloud warehouse. Fivetran connectors deploy in minutes, require zero maintenance, and automatically adjust to source changes — so your data team can stop worrying about engineering and focus on driving insights.