WhereScape was honored to team with Data Vault Inventor and Strategic Partner Dan Linstedt on a webinar recently where Linstedt clearly laid out the business case for implementing the Data Vault methodology as well as the need for automation to expedite agile Data Vault delivery and development. In the webinar entitled Accelerating Data Vault 2.0 Development with Data Warehouse Automation, Linstedt said that implementing Data Vault is a detailed and thorough process that requires sourcing the system of record, adhering to standards, rules and regulations, as well as documenting the solution. Sounds cumbersome and time-consuming, but Linstedt pointed out that Data Vault development can be achieved quickly and with agility by incorporating automation.
He cited an example of a 3-person team in Finland serving a business of 1,000 that has taken their two week sprints down to one week since incorporating automation—from development to deployment of a new Data Vault business rule.“You are not going to get 100 percent automation in the changing business rule space, but 99 percent of the data acquisition side—because it is so standardized—should be automated,” Linstedt said.
In the webinar, Linstedt talked about the cycle time of IT. “Businesses are looking to do things better, faster and cheaper, and you really can’t do that without automating or generating your solutions in your data warehousing system. In order for your information delivery team to be agile and provide the data downstream quicker, you need a metadata-driven tool…..that allows you to automate your business rules to some degree,” Linstedt said. What this leads to is simple, automatable steps—things that can be built into a process design—something that can take your metadata and very powerfully turn that into executable code.”
And what does Linstedt say to ETL developers who fear losing their job to an automation solution? Your better focus is on the business side, understanding how to apply the raw data sets to the business purpose. That is where you need to be spending your time—not on building yet another staging load.
“There is a maturity level that has been reached where automation becomes the primary focus,” Linstedt concluded. “We have to get out of that business and into the real business of data warehousing—answering business users’ questions. And the only way to do that is to automate and generate 98 percent or better of all the patterns you need to load your data warehouse, and that includes your Hadoop integration.”
If you are currently working with Data Vault, or are considering using it in an upcoming project, this webinar is not to be missed. Click here to check out the replay.