Data Warehouse Development

| April 16, 2020

Dr Barry Devlin is among the foremost authorities on business insight and one of the founders of data warehousing, having published the first architectural paper on the topic in 1988. Today he is a leading consultant and speaker on data warehouse development.

Barry has published a number of articles for WhereScape, to follow you will find a synopsis and introduction to some of these. Each has a link to the full blog so you can explore that specific subject in more detail.

Designing a Data Warehouse

Always keep in mind the basic goal of your project: to deliver a cross-functional, long-life foundation for data provision and decision support. Data warehouse development project types vary and will continue to mutate over time with requirements that you cannot predict now, and your data warehouse must continue to provide accurate data throughout this evolution.

This blog explains how to:

  • Use templates to save time and money rather than building from scratch
  • How to define and refine the logical structure of relational tables
  • Choose which approach of data modelling is best for you – 3NF, Star Schema, Data Vault etc.

Read the full blog here.

Building a Data Warehouse

This blog explains how every design is only as good as the reality of its source systems, their missing data and poorly defined data structures. The finished design is always a balance between the vision for the model and the constrains of the sources. The article covers:

  • The five steps to follow when building a data warehouse
  • How Data Warehouse Automation can help
  • How to move from design to build
  • Building a Data Vault with WhereScape Data Vault Express

Read the full blog here.

Operating a Data Warehouse

This blog explains how to deliver your data warehouse successfully to the business and run it smoothly on a daily basis. We must avoid the problems of past ad hoc data warehouse development approaches that combined manual and semi-automated methods, and adopt advanced data management and automation practices. Find out how:

  • Deployment needs to be treated as a long-term, monogamous relationship
  • To address issues such as packaging and installation of the code
  • To bundle sets of objects and transport from dev to QA and through to production
  • To handle interdependencies between the data warehouse, data marts and data lake
  • To automate the historical information that tracks performance over time

Read the full blog here.

Maintaining a Data Warehouse

In some development projects, once a piece of software is up and running it needs only minor bug-fixing, but maintaining a data warehouse needs more attention than that. The nature of creative decision-making support is that users are continuously discovering new business requirements, changing their mind about what data they need and thus demanding new data elements and structures on a weekly or monthly basis. Indeed, in some cases, the demands may arrive daily! Read this blog to find out:

  • What a data lake should and shouldn’t be used for
  • Why and how a Data Vault gives more agility in the maintenance phase
  • The role of metadata in data warehouse maintenance
  • How to predict downstream impact of changes from automated documentation

Read the full blog here.

WhereScape Announces the Release of RED 10.0.0.0

WhereScape is pleased to announce the general availability of WhereScape RED 10.0.0.0. This release is the culmination of man-years of effort. It confirms WhereScape’s commitment to continuing to develop new technologies and tools and its commitment to delivering the...

Effective AI through Data Modeling

As we journey deeper into the digital age, the importance of data modeling within the broader landscape of artificial intelligence (AI) has become more pronounced than ever. The success of AI-driven initiatives is tightly woven with the quality and structure of the...

Is Data Vault 2.0 Still Relevant?

TL;DR  Yes. Data Vault 2.0 Data Vault 2.0 is a database modeling method published in 2013. It was designed to overcome many of the shortcomings of data warehouses created using relational modeling (3NF) or star schemas (dimensional modeling). Speci fically, it...

Data Vault 2.0 Resources

Data Vault Revisited: A Six-Year Journey into the Secure Data Repository In 2017, Dr. Barry Devlin provided valuable insights about Data Vaults, a concept that sparked interest among businesses and IT professionals. Data Vaults were envisioned as secure repositories...

Understanding Data Vault 2.0

How to Avoid Pitfalls During Data Vault 2.0 Implementation Implementing a data vault as your Data Modeling approach has many advantages, such as flexibility, scalability, and efficiency. But along with that, one must be aware of the challenges that come along with...

Navigating the AI Landscape

The Pivotal Role of Data Modeling In the rapidly evolving digital age, artificial intelligence (AI) has emerged as a game-changer, deeply impacting the business landscape. Its ability to automate operations, refine decision-making processes, and significantly enhance...

Information Management Maturity

Unlocking Your Business Potential: Understanding and Enhancing Information Management Maturity In a recent report by Gartner, they emphasize the crucial role of information in the current business environment, stating, "Through 2025, organizations that are data-driven...

Data Warehousing Best Practices

In modern times, organizations are daily generating huge volumes of data. Appreciating the significance of data, companies are storing data from different departments which can be analyzed to gather insights to help the organization in better decision-making. This...

Related Content

WhereScape Announces the Release of RED 10.0.0.0

WhereScape Announces the Release of RED 10.0.0.0

WhereScape is pleased to announce the general availability of WhereScape RED 10.0.0.0. This release is the culmination of man-years of effort. It confirms WhereScape’s commitment to continuing to develop new technologies and tools and its commitment to delivering the...

WhereScape Announces the Release of RED 10.0.0.0

WhereScape Announces the Release of RED 10.0.0.0

WhereScape is pleased to announce the general availability of WhereScape RED 10.0.0.0. This release is the culmination of man-years of effort. It confirms WhereScape’s commitment to continuing to develop new technologies and tools and its commitment to delivering the...

Effective AI through Data Modeling

Effective AI through Data Modeling

As we journey deeper into the digital age, the importance of data modeling within the broader landscape of artificial intelligence (AI) has become more pronounced than ever. The success of AI-driven initiatives is tightly woven with the quality and structure of the...