We all know that data warehouses and best practices for them are changing dramatically. The global market for data warehousing is expected to grow to $34.7 billion by 2025, according to a recent report from Allied Market Research. That’s nearly double the $18.6 billion it was worth in 2017. With IoT and mobile apps generating more data than even the progenitors of “Big Data” ever thought possible, organizations find themselves needing to build new, modern data warehouses to replace the legacy systems that just can’t keep up anymore. With an estimated 175 zettabytes of data coming at us by 2025, folks are turning to cloud data warehousing in hopes of taking advantage of the performance, concurrency, simplicity, and lower cost of the cloud. With that much data coming so fast, they must find a more robust, agile, and elastic data analytics platform to keep pace and remain competitive.
So, what is a cloud data warehouse really? How is it different from traditional on-premises data warehousing and other supposed “big data” platforms? Is there really a platform that can handle the onslaught of new data and make it accessible to all the knowledge workers, analysts, and citizen data scientists in an organization?
The answer is yes! But it requires taking a brand-new approach using an innovative architecture and revolutionary new technology that is built for the cloud to serve these kinds of massive data and analytic use cases. This is not your parent’s data warehouse by any means!
Organizations using this new technology have received some mind-blowing results:
- Reducing data load time from 8 hours to 1.5 minutes
- Reducing a mission critical report’s runtime from 20+ hours to 45 minutes, with no tuning or indexes required
- Building a key fact table that now holds 39 trillion rows and is queried daily
- Building a modern data lake with over 1.1 million tables of raw data in a governed repository
- Onboarding over 11,000 users globally so they can run reports and dashboards against a single source of truth
- Natively ingesting JSON data while achieving 80% lower costs with 50x faster results
I have never seen results like this in over two decades of data warehousing. This new cloud-based data warehouse architecture has truly made data democratization a reality for many organizations. But it does not end there.
Cloud data warehouse technology has given rise to innovative systems and practices that increase efficiency and reduce costs across company functions. Today, departments like marketing, finance, and supply chain operations benefit from a modern data warehouse as much as the organization’s engineering and data science teams.
There are five business priorities that are continuing to fuel increased investment in modern enterprise data warehouses in the cloud:
- The need to access and act on data in real time
- The continued search for a holistic view of the customer (across new data sets and types)
- Understanding data lineage to ensure regulatory compliance
- Enabling non-technical people to query data quickly and cheaply
- The need to bring all the data together into a single, governed, location
With the advances in cloud data warehouse technology, these and many other critical business use cases can now be solved, usually for a much lower TCO. If this is coupled with advanced, metadata-driven, data warehouse automation software like WhereScape® automation for Snowflake, organizations can achieve even further cost reductions as well as speeding the time to value of all the organization's data.
To learn more about the reality of cloud data warehousing, be sure to attend my session Changing the Game with Cloud Data Warehousing at the upcoming WhereScape Connect conference in Denver, Colorado where I will show you the reality and how you can derive more value from your data using Snowflake’s elastic data platform that was built specifically for the cloud. The session will be held on Thursday, October 10 at 9:45 am.
Not yet registered to join us October 8 – 10th at WhereScape Connect? You can learn more about WhereScape’s North America customer conference and register here.
About the author:
Kent Graziano is the Chief Technical Evangelist at Snowflake. His is an award-winning author, speaker, and trainer, in the areas of data modeling, data architecture, and data warehousing. He is an Oracle ACE Director - Alumni, member of the OakTable Network, a certified Data Vault Master and Data Vault 2.0 Practitioner (CDVP2), expert data modeler and solution architect with more than 30 years of experience, including over two decades doing data warehousing and business intelligence (in multiple industries). He is an internationally recognized expert in data modeling and Agile Data Warehousing. Mr. Graziano has developed and led many successful software and data warehouse implementation teams, including multiple agile DW/BI teams. He has written numerous articles, authored three Kindle books (available on Amazon.com), co-authored four books (including the 1st Edition of The Data Model Resource Book), and has given hundreds of presentations, nationally and internationally. He was a co-author on the first book on Data Vault, and the technical editor for Super Charge Your Data Warehouse. You can follow Kent on twitter @KentGraziano or on his blog The Data Warrior (http://kentgraziano.com).