Deploy
Deployment: Hot Start, Or Hard Work?
Deploying a designed and successfully-prototyped data warehouse is typically hard work.
The design has been completed (and perhaps even tested) and signed off. But standing between you and sign off - and more importantly between your now-expectant user community and the gratification of their need for decisional data - are:
- workflow and data flow mechanisms
- data movement, loading and staging logic, and the concurrency and recovery mechanisms you need to handle invalid data and extraction failures
- last-minute changes, enhancements and additions to the prototype
- documentation for end-users and operations personnel and a half-dozen other hurdles.
Often designers exit the project at this stage to be replaced by programmers and database administrators who try to back the tractor-trailer-sized extraction, transformation and load (ETL) tool of choice up into the space between the data warehouse and the production system, and map what was often an elegant and effective logical design into what will become a highly derivative physical implementation of that design....weeks or months later.
On occasion the data warehouse team begins the laborious, time-consuming but usually ineffective task of producing source- and target-system-specific procedural code required to map the source system data into a fairly exact physical implementation of the logical schema design....which comes online months or sometimes years later, to serve a user constituency that now has completely different analytical needs.
There are two reasons why the seam between the end of the design phase and the deployment-complete milestone is the graveyard zone for data warehouse projects: this is the space where one set of (often crude and repurposed) tools hand off logical designs to another set of (over-engineered, general purpose data movement) tools, and where one kind of thinking - the designer's thinking - now has to meet, modify and be modified by another kind of thinking: the system manager's thinking. Crossing that chasm is, indeed, hard work.
Unless, of course, there is no seam, no chasm, to cross.
With WhereScape RED, all the essential deployment functionality is integrated into the design and prototyping facilities, and has already been used, in prototype. Once the design team has a prototype that meets requirements, WhereScape RED automatically generates tables, indices and procedures, complete within the target operational environment.
For simple, one- or two-source single schema data warehouses or data marts, this is the much-talked-out and rarely-seen hot start - a signed-off prototype that is in production, with full historical population at the lowest level of transactional detail required, the next day. Literally. No ETL to maneuver into the architecture -- it's already there. No code to write - it will write itself, as it has done for days or weeks past. No scheduler to configure: WhereScape manages the extraction schedule, and handles rollback-on-failure scenarios and dependencies.
WhereScape RED generates native database procedural code that is easy to read and modify. We even provide a graphical editor and compiler. Code can be checked out, delete locked, compared to the code in the database and, if you like, compared to previous versions. Different objects in the design will have appropriate, customized code generated for them. For dimensions, designers can choose between Type 1, Type 2 or Type 3 or even Type 4 (externally managed) slowly-changing dimension code. For staging tables, designers can decide to do cursor or set-based updates. And, as in design and prototyping, WhereScape RED continues to enforce the key elements of sound data warehouse methodology, generating surrogate keys - automatically of course - by default. Finally, WhereScape RED generates all relevant indexes for the production data warehouse, and then allows savvy designers and DBAs to modify those index creation operations as they see fit.
And documentation? Documentation is generated automatically, at any stage you choose to generate it.
WhereScape RED automatically produces HTML-based technical and user documentation: schema diagrams, track-back diagrams for data lineage and for the procedural code, and complete user documentation including a glossary of terms used in the data warehouse.
Unconvinced? Check us out more and become a convert -- Download a copy of WhereScape RED now and take us for a spin.