TDWI San Diego has kicked off, and the conference theme is “Big Data”. TDWI seems to like themes – Boston in September is “Agile” (of course we will be at that one), and Orlando in November is “Emerging Technologies”.
You can catch us at TDWI San Diego. We have a booth where we are catching up with customers, analysts and press, as well as introducing our software to prospective users. One of the questions we get asked, along with “Can you really help me build a data warehouse that quickly?” (Yes), “I use Teradata/SQL Server/Oracle/DB2 can you still help me?” (Yes), “Are you an ETL tool?” (No, but let’s sit down and talk about what you need to achieve rather than what software category you want) is often “So what is WhereScape’s view on Big Data?”
I have enjoyed listening to Colin White talk about Big Data at the Masters of BI events that WhereScape have run, have listened intently to Colin and Harriet Fryman from IBM present and discuss Big Data Analytics at the Pacific Northwest BI Summit and I have even been on a panel talking about Big Data at a CIO Magazine event. I have had discussions with customers, analysts and other vendors, and have also listened to webinars, and read widely on the topic (one of the best pieces I have read is from Mark Madsen previewing the keynote he and Marc Demarest are giving at TDWI San Diego ).
Defining Big Data
I am not convinced there is a common understanding of what Big Data is. I will leave it to the analysts to come up with the perfect definition. For the moment I am going with:
Data that exceeds the capability of conventional databases and approaches.
Mark Madsen wrote:
“What we're facing is an architectural shift. You can think of it like the Web 1.0 to Web 2.0 shift. Web 1.0 was largely read-only, publishing oriented, just like BI and the data warehouse. Web 2.0 is read-write, peer-to-peer at both a human and machine level. That was an architecture shift as much as a technology shift. We reconfigured the components in the architecture and created different ways to build and scale software.”
I really like the idea that Big Data is an architectural shift.
So what is WhereScape’s view on Big Data?
WhereScape Software is a data warehousing software company. We believe traditionally data warehouses take too long to build, and once built are too hard to change. Despite these shortcomings, we passionately believe that data warehouses have a role within data architectures. But data warehouses will be different. Already many of our customers do not view their data warehouse as a heterogeneous platform – they no longer have a Teradata or SQL Server data warehouse; our customers now have a data warehouse that includes Teradata or SQL Server.
Our view is that big data redefines the role and usage of data within an organization. Some things do not change – there is still absolutely a place for a well-managed, well governed data warehouse (just built quicker, and more responsive to change, thank you very much). But with Big Data we now need to think about all the other data that an organization could use for business benefit – but that will never make it into the traditional data warehouse, or will only be included in an aggregated or derived form. This data could be in an appliance, a non-relational store, external or streamed. As data people we need to be cognizant of changes in what our user constituency define as useful data.
WhereScape RED in a Big Data World
When we announced WhereScape RED 7, Shawn Rogers, Vice President of Research, Business Intelligence and Data Warehousing at Enterprise Management Associate wrote:
“The days of the enterprise data warehouse operating as the centralized hub for all reporting and analytics is a thing of the past. Database appliances are gaining traction as IT organizations warm to the notion of using purpose-built platforms for specific analytic tasks, but the speed of the platforms has naturally outpaced the productivity tools needed to support them.
WhereScape RED 7 is ideally suited for today’s highly diverse, special-purpose IT infrastructures. WhereScape has increased its value proposition by moving from ‘let us build and manage your data warehouse’ to ‘let us manage your extended data ecosystem, which includes your data warehouse’”.
Big data may change how we view a data warehouse and data within an organization, but the concepts of time to value and responsive to change remain, and that is where WhereScape RED continues to be focused. Bring on Big Data, and bring on WhereScape RED.