Snowflake Guest Blog: Cloud Data Warehousing – It’s Now & It’s Real!

| September 9, 2019
Snowflake Guest Blog: Cloud Data Warehousing – It’s Now & It’s Real!

We all know that data warehouses and best practices for them are changing dramatically. The global market for data warehousing is expected to grow to $34.7 billion by 2025, according to a recent report from Allied Market Research. That’s nearly double the $18.6 billion it was worth in 2017. With IoT and mobile apps generating more data than even the progenitors of “Big Data” ever thought possible, organizations find themselves needing to build new, modern data warehouses to replace the legacy systems that just can’t keep up anymore. With an estimated 175 zettabytes of data coming at us by 2025, folks are turning to cloud data warehousing in hopes of taking advantage of the performance, concurrency, simplicity, and lower cost of the cloud. With that much data coming so fast, they must find a more robust, agile, and elastic data analytics platform to keep pace and remain competitive.

So, what is a cloud data warehouse really? How is it different from traditional on-premises data warehousing and other supposed “big data” platforms? Is there really a platform that can handle the onslaught of new data and make it accessible to all the knowledge workers, analysts, and citizen data scientists in an organization?

The answer is yes! But it requires taking a brand-new approach using an innovative architecture and revolutionary new technology that is built for the cloud to serve these kinds of massive data and analytic use cases. This is not your parent’s data warehouse by any means!

Organizations using this new technology have received some mind-blowing results:

  • Reducing data load time from 8 hours to 1.5 minutes
  • Reducing a mission critical report’s runtime from 20+ hours to 45 minutes, with no tuning or indexes required
  • Building a key fact table that now holds 39 trillion rows and is queried daily
  • Building a modern data lake with over 1.1 million tables of raw data in a governed repository
  • Onboarding over 11,000 users globally so they can run reports and dashboards against a single source of truth
  • Natively ingesting JSON data while achieving 80% lower costs with 50x faster results

I have never seen results like this in over two decades of data warehousing. This new cloud-based data warehouse architecture has truly made data democratization a reality for many organizations. But it does not end there.

Cloud data warehouse technology has given rise to innovative systems and practices that increase efficiency and reduce costs across company functions. Today, departments like marketing, finance, and supply chain operations benefit from a modern data warehouse as much as the organization’s engineering and data science teams.

There are five business priorities that are continuing to fuel increased investment in modern enterprise data warehouses in the cloud:

  1. The need to access and act on data in real time
  2. The continued search for a holistic view of the customer (across new data sets and types)
  3. Understanding data lineage to ensure regulatory compliance
  4. Enabling non-technical people to query data quickly and cheaply
  5. The need to bring all the data together into a single, governed, location

With the advances in cloud data warehouse technology, these and many other critical business use cases can now be solved, usually for a much lower TCO. If this is coupled with advanced, metadata-driven, data warehouse automation software like WhereScape® automation for Snowflake, organizations can achieve even further cost reductions as well as speeding the time to value of all the organization’s data.

To learn more about the reality of cloud data warehousing, be sure to attend my session Changing the Game with Cloud Data Warehousing at the upcoming WhereScape Connect conference in Denver, Colorado where I will show you the reality and how you can derive more value from your data using Snowflake’s elastic data platform that was built specifically for the cloud. The session will be held on Thursday, October 10 at 9:45 am.

Not yet registered to join us October 8 – 10th at WhereScape Connect? You can learn more about WhereScape’s North America customer conference and register here.

About the author:

Kent Graziano is the Chief Technical Evangelist at Snowflake. His is an award-winning author, speaker, and trainer, in the areas of data modeling, data architecture, and data warehousing. He is an Oracle ACE Director – Alumni, member of the OakTable Network, a certified Data Vault Master and Data Vault 2.0 Practitioner (CDVP2), expert data modeler and solution architect with more than 30 years of experience, including over two decades doing data warehousing and business intelligence (in multiple industries). He is an internationally recognized expert in data modeling and Agile Data Warehousing. Mr. Graziano has developed and led many successful software and data warehouse implementation teams, including multiple agile DW/BI teams. He has written numerous articles, authored three Kindle books (available on Amazon.com), co-authored four books (including the 1st Edition of The Data Model Resource Book), and has given hundreds of presentations, nationally and internationally. He was a co-author on the first book on Data Vault, and the technical editor for Super Charge Your Data Warehouse. You can follow Kent on twitter @KentGraziano or on his blog The Data Warrior (http://kentgraziano.com).

How to Hire and Retain Data Warehouse Developers

The projected data warehouse developer job growth rate is 21% from 2018-2028, with about 284,100 new jobs for data warehouse developers projected over the next decade, according to Zippia. This surge in demand for data warehouse talent is being felt across businesses...

8 Reasons to Make the Switch to ELT Automation

Extraction, loading, and transformation (ELT) processes have been in existence for almost 30 years. It has been a programming skill set mandatory for those responsible for the creation of analytical environments and their maintenance because ELT automation works....

What is a Data Model?

A data model depicts a company's data organization, standardizing the relationships among data elements and their correspondence to real-world entities' properties. It facilitates the organization of data for business processes and information systems, offering tools...

Webinar Recap: Navigating the Future of Data Analytics

In an era where data is the new gold, understanding its trajectory is crucial for any forward-thinking organization. Our recent webinar, "Capitalizing on Data Analytic Predictions by Focusing on Cross-Functional Value of Automation and Modernization," hosted in...

Introducing: Data Automation Levels

The concept of automation has seamlessly integrated into many aspects of our lives, from self-driving cars to sophisticated software systems. Recently, Mercedes-Benz announced their achievement in reaching Level 3 in automated driving technology, which got me thinking...

Agile Data Warehouse Design for Rapid Prototyping

Agile Prototyping: Revolutionizing Data Warehouse Design While most people know WhereScape for its automated code generator that eradicates repetitive hand-coding tasks, there is another major way in which the software can save huge amounts of time and resources....

Related Content

How to Hire and Retain Data Warehouse Developers

How to Hire and Retain Data Warehouse Developers

The projected data warehouse developer job growth rate is 21% from 2018-2028, with about 284,100 new jobs for data warehouse developers projected over the next decade, according to Zippia. This surge in demand for data warehouse talent is being felt across businesses...

8 Reasons to Make the Switch to ELT Automation

8 Reasons to Make the Switch to ELT Automation

Extraction, loading, and transformation (ELT) processes have been in existence for almost 30 years. It has been a programming skill set mandatory for those responsible for the creation of analytical environments and their maintenance because ELT automation works....

How to Hire and Retain Data Warehouse Developers

How to Hire and Retain Data Warehouse Developers

The projected data warehouse developer job growth rate is 21% from 2018-2028, with about 284,100 new jobs for data warehouse developers projected over the next decade, according to Zippia. This surge in demand for data warehouse talent is being felt across businesses...

8 Reasons to Make the Switch to ELT Automation

8 Reasons to Make the Switch to ELT Automation

Extraction, loading, and transformation (ELT) processes have been in existence for almost 30 years. It has been a programming skill set mandatory for those responsible for the creation of analytical environments and their maintenance because ELT automation works....

What is a Data Model?

What is a Data Model?

A data model depicts a company's data organization, standardizing the relationships among data elements and their correspondence to real-world entities' properties. It facilitates the organization of data for business processes and information systems, offering tools...