Tesco shines spotlight on agile data warehouse overhaul
Tesco's Andy Ruckley talks about how agile development with help from Teradata helped resolve numerous big data issues
Tesco has been in the headlines for the wrong reasons as of late, with profit warnings and the recent revelation that it had made inaccurate sales forecasts to the tune of hundreds of millions.
However, that didn't seem to take the gloss off a polished performance by the company's head of technology for data and platforms at this week's Teradata Partners user conference in Nashville.
A packed audience listened to Andy Ruckley as he described how the company has now got its data warehouse in order over the last few months after it was found that it was failing the business.
Ruckley became a leader of the company's big data efforts last year and he told delegates how the state of the company's previous data warehouse meant Tesco's data service "could not keep up with the pace of change".
Ruckley said reports were often delayed, data quality was hampered by hand coded business rules, there was inconsistent data quality, data architecture remained in silos instead of being integrated, and there was a lack of full disaster recovery for data.
As one of the world's largest retailers this might be quite disturbing to hear for Tesco's embattled board and increasingly angry Tesco shareholders, but, nethertheless, Ruckley told the conference: "What I discovered is that we could get from A to B with our data [or luggage as he described it] but it was inefficient and cumbersome."
Tesco experienced its problems despite being an established Teradata user. Ruckley said: "My Teradata account manager said 'buy a bigger box'. My predecessor bought a bigger box but it didn't address the root cause of the problems and I wouldn't be addressing my customers' [IT users] needs."
Last July, Ruckley said he asked his team to come up with a plan to launch a "big bang" solution to the problems, but was told it would still take three years to complete. As Ruckley explained in response to this news: "I might as well have resigned there and then".
Instead, after attending last year's Teradata Partners conference he brought in a Teradata professional services team alongside a developer team from Teradata partner WhereScape, a company that sells a solution to make it easier to model, write and produce code, and who he found out about at the conference.
The project was started in earnest this spring, with Teradata and WhereScape helping Ruckley complete an agile warehouse development project to cover nine key business areas: sales, products, customers, prices, promotions, buyers, brands, calendars and locations.
The six-month project has now been completed, smashing the previous three-year forecast by using agile development to overhaul each data warehouse business area in "sprints", as Ruckley described the modular and agile data development efforts.
Tesco brought in developers, data analysts and architects from Teradata and WhereScape for the duration, with Ruckley saying his own team has now learnt from them. "We were the first in the Tesco IT function to do agile, and it was an exciting way of working for my team - it is rewarding for creative people," he said.
Reference: Computer World UK