Tune in for a live WhereScape 3D + DVE virtual...
Living in a Data Vault
Living in a vault, especially of the underground variety, doesn’t sound too pleasant to me. However, a vault can also mean a roof in the form of an arch, as in a cathedral. To that point, living in a Data Vault demands some over-arching thought, uniting business people and IT once again in a process of ongoing decision-making support, in an environment where business change will always be continuous and rapid.
Such a constant “running while rebuilding” process is well-known to data warehouse teams, but the support for agility combined with the governance offered by the Data Vault model and methodology raises the stakes to unprecedented levels. Getting business people on board, as described in my past blogpost Breaking into a Data Vault is the first and vital step. Business users must see that the agility they experienced in the initial delivery of a Data Vault—when implemented via data warehouse automation software such as WhereScape® Data Vault Express—can carry through to ongoing operation.
The collaborative effort between business users and IT in the discovery and design phase of the initial Data Vault delivery must continue. And intensify at an even faster pace. Business users may have had some patience with the initial build of the Data Vault because of its novelty, but now that the Data Vault is mainstream, the expectation will be of ever faster and higher quality iterations. The metadata stored in WhereScape Data Vault Express is the basis for impact analysis, rapid development and speedy deployment of Data Vault updates. Ensuring updates are delivered successfully the first time—in terms of data delivered and business needs met—will keep business users from reverting to the omnipresent spreadsheet.
For development teams, the automation of the entire process from design to delivery, as discussed in the blogpost Locking in a Data Vault, is the mandatory foundation for reducing the effort of 24/7 operation and freeing up developer time and resources for the delivery of new and changed function and data demanded by the business. The error handling, logging and auditing code generated in the development phase together with the built-in scheduler and job management capabilities of WhereScape Data Vault Express allow developers to focus on what they do best: develop solutions.
Agility is vital in this extended stage of the Data Vault lifecycle. This flexibility comes at the cost of a relatively complex engineering structure in the Data Vault model as well as in the numerous components and processes needed to build and populate it. While Data Vault 2.0 offers a methodology to address this complexity, data warehouse automation, and the metadata it creates and uses, is the only effective solution for developers under pressure from rapidly evolving business demands.
This metadata—or, as I prefer to call it, context-setting information—at the heart of WhereScape Data Vault Express offers a common and consistent vocabulary and business semantics. These are mandatory for effective collaboration between business and IT as business evolves, sometimes dramatically, and where the content and function of the Data Vault must follow. Manual documentation of business needs or data deliverables cannot be trusted in such circumstances. Metadata, in effect, is the automated and flexible documentation of all requirements and deliverables.
Living in the Data Vault is the final and ongoing phase of the Data Vault delivery process. It deserves more attention than it usually receives. After all, initial implementation—like a puppy—is for one short time period, a Data Vault is for life. To set your organization up for greater success in implementing and living within the Data Vault methodology, read this Meeting the Six Data Vault Challenges white paper and watch this recent Turning the Data Vault Methodology into a Practical Reality recorded webcast.
You can find the other blog posts in this series here:
Dr. Barry Devlin is among the foremost authorities on business insight and one of the founders of data warehousing, having published the first architectural paper on the topic in 1988. Barry is founder and principal of 9sight Consulting. A regular blogger, writer and commentator on information and its use, Barry is based in Cape Town, South Africa and operates worldwide.
Guide to Data Quality: Ensuring Accuracy and Consistency in Your Organization
Why Data Quality Matters Data is only as useful as it is accurate and complete. No matter how many analysis models and data review routines you put into place, your organization can’t truly make data-driven decisions without accurate, relevant, complete, and...
Common Data Quality Challenges and How to Overcome Them
The Importance of Maintaining Data Quality Improving data quality is a top priority for many forward-thinking organizations, and for good reason. Any company making decisions based on data should also invest time and resources into ensuring high data quality. Data...
What is a Cloud Data Warehouse?
As organizations increasingly turn to data-driven decision-making, the demand for cloud data warehouses continues to rise. The cloud data warehouse market is projected to grow significantly, reaching $10.42 billion by 2026 with a compound annual growth rate (CAGR) of...
Developers’ Best Friend: WhereScape Saves Countless Hours
Development teams often struggle with an imbalance between building new features and maintaining existing code. According to studies, up to 75% of a developer's time is spent debugging and fixing code, much of it due to manual processes. This results in 620 million...
Mastering Data Vault Modeling: Architecture, Best Practices, and Essential Tools
What is Data Vault Modeling? To effectively manage large-scale and complex data environments, many data teams turn to Data Vault modeling. This technique provides a highly scalable and flexible architecture that can easily adapt to the growing and changing needs of an...
Scaling Data Warehouses in Education: Strategies for Managing Growing Data Demand
Approximately 74% of educational leaders report that data-driven decision-making enhances institutional performance and helps achieve academic goals. [1] Pinpointing effective data management strategies in education can make a profound impact on learning...
Future-Proofing Manufacturing IT with WhereScape: Driving Efficiency and Innovation
Manufacturing IT strives to conserve resources and add efficiency through the strategic use of data and technology solutions. Toward that end, manufacturing IT teams can drive efficiency and innovation by selecting top tools for data-driven manufacturing and...
The Competitive Advantages of WhereScape
After nearly a quarter-century in the data automation field, WhereScape has established itself as a leader by offering unparalleled capabilities that surpass its competitors. Today we’ll dive into the advantages of WhereScape and highlight why it is the premier data...
Data Management In Healthcare: Streamlining Operations for Improved Care
Appropriate and efficient data management in healthcare plays a large role in staff bandwidth, patient experience, and health outcomes. Healthcare teams require access to patient records and treatment history in order to properly perform their jobs. Operationally,...
WhereScape 3D 9.0.4 Now Available: Integrate with Microsoft Purview
We are excited to announce the release of WhereScape 3D Version 9.0.4, which is packed with new enhancements, highlighted by the integration with Microsoft Purview. Additional features include advanced data profiling for custom connections, Pebble extensions for...
Related Content
Guide to Data Quality: Ensuring Accuracy and Consistency in Your Organization
Why Data Quality Matters Data is only as useful as it is accurate and complete. No matter how many analysis models and data review routines you put into place, your organization can’t truly make data-driven decisions without accurate, relevant, complete, and...
Common Data Quality Challenges and How to Overcome Them
The Importance of Maintaining Data Quality Improving data quality is a top priority for many forward-thinking organizations, and for good reason. Any company making decisions based on data should also invest time and resources into ensuring high data quality. Data...
What is a Cloud Data Warehouse?
A cloud data warehouse is an advanced database service managed and hosted over the internet.
Developers’ Best Friend: WhereScape Saves Countless Hours
Development teams often struggle with an imbalance between building new features and maintaining existing code. According to studies, up to 75% of a developer's time is spent debugging and fixing code, much of it due to manual processes. This results in 620 million...