If you say the words Novopay in New Zealand everyone knows what you are talking about – a classic failure of an IT system. Novopay is an end to end payroll service provided to the New Zealand Ministry of Education by an Australian firm called Talent2. It is the most complex payroll in the country, paying 110,000 people employed in 2500 schools and covers 15 collective agreements, but has been beset by problems since going live in August 2012.
Being a government project there is a fascinating amount of detail available if you want to witness a train wreck in slow motion. The risk, issues and monitoring reports are all available online.
To give some idea of the scale and implications of continued problems the New Zealand Government has created the position of Minister Responsible for Novapay. They have given this (potential hospital pass) to their trusted fix-up person, Hon. Steven Joyce.
The disaster has created a feeding frenzy for the consultants – Deloitte have undertaken a technical review, PwC have been engaged to report on individual pay periods, and a new dedicated Education Sector Payroll Services business unit (headed by former Inland Revenue Deputy Commissioner and KiwiSaver Programme Director Cathy Magiannis) has been established in the Ministry of Education to co-ordinate the Remediation Plan.
I can’t offer any specific insight into payroll systems and have no experience in the area (although I do actually recall visiting Concept Systems the original developers of the Alesco system at the heart of Novopay many many years ago). However I have had involvement with hundreds of IT projects over the years.
To me this looks like a failure in the procurement process and a failure of the project approach.
This is, by New Zealand standards, a complex system. The current tender system used by the New Zealand government and other large entities is the broken step in the process. I can’t speak for this particular process, but I have seen and been involved in enough government Request for Information/Request for Proposal to know they don’t work. There is no way to know what purchaser can specify the requirements accurately, and as can be seen clearly in this case, the idea of contracting out of risk does not work as a defence when you are on the 6 o’clock news. While I would love to see government procurement changed, it is not an area that I personally care to spend any time on.
Many aspects of the project failure have relevance in a data warehousing, and even more so in the “big data” world. The Novopay project was beset by complexity and scope problems. This is the norm in the data world as we are never in control of the data we receive.
How do you mitigate this? At WhereScape we recommend late learning strategies – something the agile community also champions. We recommend only going as far as basic structures until we have confidence in what we are doing. As an example, if you are building a dimensional data warehouse we recommend building to first level fact tables initially, and then socializing the results. Hold off on fancy aggregates, complex cubes and beautiful dashboards and reports as long as possible as the work can be wasted if we are delivering the wrong types of data.
This is even more important in a big data world where it can take far longer to get an idea of what is possible when you are interacting with large complex data sets as opposed to (relatively) well behaved source systems.
Projects are less risky when you identify as many risk factors as early as possible. We designed WhereScape 3D to specifically help users answer the biggest questions around scope and approach – which in our world is to do with data. Given it is a constraint in every project it needs to be examined and dealt with up front, not left as a surprise in each and every project we do.
Novopay knew that the payroll was complex, there was clearly a lot they did not know, and probably a lot they could not know until late in the project. Wouldn't it have been better to know that sooner?