Why Automating Snowflake is a Huge Deal for Agile Data Warehousing

| August 30, 2017

Sometimes a major technological advance is accelerated by another complementary innovation. It seems data automation and a Cloud-native database that separates storage from compute go together just like peas and carrots. Two huge steps forward in data management and democratization have just become one giant leap, and here’s why…

Ten years ago, could you imagine a giant 3D dragon jumping out of your bedroom wall? The rate of technological advance means you can buy a 3D projector the size of a handbag for $500 these days, whereas a few years ago this technology was only accessible in a cinema. First, you had big standard definition home projectors that made lots of noise and got very hot; then smaller, quieter HD; then 3D and now Ultra HD and so on. The gradual nature of this feels natural as everyone adjusts to the latest model, then suddenly you have a dragon leaping out of your wall and that feels normal.

Sometimes technology lurches forward in a giant step, skipping those granular stages. This shift precedes the understanding and acceptance of it, requiring people to retrospectively figure out how they can use its power. Breakthroughs like Big Data, which exploded with Hadoop’s ability to process huge streams of data in parallel, are solutions looking for problems: many people want Big Data strategies and invest in it without knowing exactly what data they want and why they really need it.

Scratching a 30-Year Itch

Snowflake is different, as it’s an answer to a problem IT has suffered for too long. In the ‘90s data warehouses and the resultant business insight were supposed to be the holy grail that would transform how we do business. However, the logistics of building and managing a data warehouse that worked were more complex than we had first hoped. So, the power of data resided with a few specialists, its extraction in any meaningful form was slow, and this created political struggles within organizations that disabled rather than enabled the majority.

In the beginning, on-premises data warehouses were hugely expensive and inhibited agility. Building a data warehouse took years and cost millions, meaning only the richest companies could afford one. We had to estimate how much storage and compute power we needed three years in advance. Buy too little and we ran out of space and lost the ability to do our job, buy too much and we wasted huge amounts of money on unused memory.

The Cloud changed this by allowing us to only pay for what we need. Now with increased security and trust, we are shifting more critical workloads on to the Cloud, but we can still often spend more money than is needed. While storage is cheap, compute power is expensive and often VMs sit unused, racking up huge bills, money that can now be spent on growing the business elsewhere. People often talk of the Cloud being expandable to grow with a company based on their needs, but its ability to contract so we only spend what we need to is equally, if not more, important.

The Snowflake Effect

Snowflake has separated storage and compute, and the bar has been raised again. As Snowflake’s VP of Product and Partner Marketing, Jon Bock, explains on this podcast, if we just want to drive to the shops and back, we don’t need to buy a Ferrari, but if we want to go for a long drive on the open road, it would be nice to rent one cheap. This means whole teams only have to pay for the specific amount of computing power each individual needs at the time they need it, while also allowing all team members to access and collaborate on a huge pool of shared storage data.

Snowflake is a Cloud native database. Whether you’re a Cloud native company or not, Snowflake enables you to take advantage of this latest technology and run like a digital native, only paying for the storage and compute you need at any time. When we heard about what they were doing, we knew it complemented our offering perfectly. WhereScape automation makes the design, development, deployment, and operation of data warehouses and other structures quicker and cheaper, so teams can deliver projects in hours and days, not months and years. Snowflake allows IT teams to spin up new data environments in the Cloud in seconds.

WhereScape® automation for Snowflake represents one of those shifts that disrupts the whole geology of IT, a seismic shift that answers the new holy grail of business requirements – doing more with less and delivering faster. It also makes the availability of cheap, efficient data warehouses possible to companies of all sizes, and takes them out of the hands of experts to democratize data in a way we haven’t yet experienced. It’s time to make dragons leap out of your wall.

This blog is based on an Inside Analysis podcast in which Snowflake VP of Product and Partner Marketing Jon Bock and WhereScape CEO Mark Budzinski discuss the potential of data warehouse automation for the Cloud. Click here to listen to the podcast.

WhereScape Announces the Release of RED 10.0.0.0

WhereScape is pleased to announce the general availability of WhereScape RED 10.0.0.0. This release is the culmination of man-years of effort. It confirms WhereScape’s commitment to continuing to develop new technologies and tools and its commitment to delivering the...

Effective AI through Data Modeling

As we journey deeper into the digital age, the importance of data modeling within the broader landscape of artificial intelligence (AI) has become more pronounced than ever. The success of AI-driven initiatives is tightly woven with the quality and structure of the...

Is Data Vault 2.0 Still Relevant?

TL;DR  Yes. Data Vault 2.0 Data Vault 2.0 is a database modeling method published in 2013. It was designed to overcome many of the shortcomings of data warehouses created using relational modeling (3NF) or star schemas (dimensional modeling). Speci fically, it...

Data Vault 2.0 Resources

Data Vault Revisited: A Six-Year Journey into the Secure Data Repository In 2017, Dr. Barry Devlin provided valuable insights about Data Vaults, a concept that sparked interest among businesses and IT professionals. Data Vaults were envisioned as secure repositories...

Understanding Data Vault 2.0

How to Avoid Pitfalls During Data Vault 2.0 Implementation Implementing a data vault as your Data Modeling approach has many advantages, such as flexibility, scalability, and efficiency. But along with that, one must be aware of the challenges that come along with...

Navigating the AI Landscape

The Pivotal Role of Data Modeling In the rapidly evolving digital age, artificial intelligence (AI) has emerged as a game-changer, deeply impacting the business landscape. Its ability to automate operations, refine decision-making processes, and significantly enhance...

Information Management Maturity

Unlocking Your Business Potential: Understanding and Enhancing Information Management Maturity In a recent report by Gartner, they emphasize the crucial role of information in the current business environment, stating, "Through 2025, organizations that are data-driven...

Data Warehousing Best Practices

In modern times, organizations are daily generating huge volumes of data. Appreciating the significance of data, companies are storing data from different departments which can be analyzed to gather insights to help the organization in better decision-making. This...

Related Content

WhereScape Announces the Release of RED 10.0.0.0

WhereScape Announces the Release of RED 10.0.0.0

WhereScape is pleased to announce the general availability of WhereScape RED 10.0.0.0. This release is the culmination of man-years of effort. It confirms WhereScape’s commitment to continuing to develop new technologies and tools and its commitment to delivering the...

WhereScape Announces the Release of RED 10.0.0.0

WhereScape Announces the Release of RED 10.0.0.0

WhereScape is pleased to announce the general availability of WhereScape RED 10.0.0.0. This release is the culmination of man-years of effort. It confirms WhereScape’s commitment to continuing to develop new technologies and tools and its commitment to delivering the...

Effective AI through Data Modeling

Effective AI through Data Modeling

As we journey deeper into the digital age, the importance of data modeling within the broader landscape of artificial intelligence (AI) has become more pronounced than ever. The success of AI-driven initiatives is tightly woven with the quality and structure of the...