Operate
Data Warehouse Operations: Unglamorous Mission-Criticality
Put together a good logical design. Prototype that design. Test it with users, iterate, and test again. Modify the design substantially after new user input. Populate the new prototype. Get user design approval. Take the design into production. You're a hero.
To everyone but the poor sap who's been stuck with hazardous mission of managing the cobbled-together, creaking-and-groaning mass of hardware, software and data you call a data warehouse, and they call "one of the primary reasons for my decision to seek employment elsewhere." You're the hero who gave the finance department the data they needed for that big end-of-quarter push. They are the loser who can't keep your data warehouse up and running.
The design, prototype and deployment phases are finite - they end, and end sooner and more successfully with WhereScape RED. But data warehouse operation lasts forever. Mission-critical and unglamorous, operational stability makes or breaks the initially-successful data warehouse project - an enterprise that relies on data to survive comes to expect, almost unconsciously, that the data warehouse will be up, complete, loaded with reasonable performance when required. Whenever required. When the load set is large, taking hours to extract, move, stage and load, you have one chance to get nightly refresh or drop-and-reload strategies right.... and no second chances to win back a user community that came in on Tuesday morning to find the data warehouse down, or, worse, populated with unreliable data.
WhereScape RED helps operations and management personnel keep production data warehouses stable and available in four significant ways:
- by enabling multiple prototype-and-iterate cycles during early phases of the data warehouse design processes, the extraction-to-availability cycle is debugged when it should be debugged: during design, and not in production
- by auto-generating source- and target-specific code, WhereScape RED's code is of higher quality than hand-crafted extraction
- by providing a scheduler as an integrated component of WhereScape RED, we provide all the mechanisms required for management, restarting and full auditing
- by utilizing the inherent power of the database rather than adding another transformation engine, lowering maintenance (and licence) costs
WhereScape's integrated data warehouse scheduler can run on most UNIX variants (including major variants of LINUX) or as a System Service under compatible Microsoft Windows operating systems. It can be called from an enterprise scheduler, or can provide stand alone data warehouse scheduling capabilities.
WhereScape's database and job monitoring (Oracle only) allows individual jobs to be tracked and alerts raised when jobs perform abnormally: when they take too long, or don't start on time or have too many errors.
Dubious? Operate one yourself -- Download a copy of WhereScape RED now and find out what it's like to spend a few minutes a week managing your data warehouse. And what it's like to sleep well at night.