Earlier this month I presented at the Gartner Data & Analytics Summit in London, discussing how businesses can adapt to the analytics and infrastructure requirements that modern business now demands. I was thrilled to see the room packed with those interested in leveraging automation for competitive advantage today. A striking uptick in interest compared to what the topic would have attracted only a few years back, leaving me to consider what has contributed to that shift.
The fact of the matter is that even just five years ago we were in a much simpler world when it came to data. Most data being used for analytics was stored in a SQL database, and could be queried easily to deliver insights. But that simplicity is now a thing of the past. The type of analytics many companies are wanting to do today – data science, machine learning, etc – and the sheer volume, velocity or variety of data businesses use and collect today combine to mean the IT environments and infrastructures we used back then undoubtedly cannot cope any longer. Now, individual departments find themselves with multiple databases, different data streams in different formats, as well as faster turn-around times for delivering mission-critical insight to the business function from all of these disparate sources. The companies which can succeed with turning around the best insights the fastest, are the ones that will thrive in the Information Era.
In response to this, and the changing information landscape, businesses are embarking on ambitious infrastructure modernization and digital transformation projects. At the core of many of these is embracing a cloud data infrastructure, which can help mitigate the rising costs of data storage, with lower management requirements, provide greater elasticity given demand, and enable wider access for easier data sharing.
However, cloud environments don’t build themselves – and often IT teams are trying to build the entire operation from the ground up. This is an error-prone, laborious process and riddled with data governance and management concerns if done incorrectly. So, what’s the solution?
Well, the good news is that alongside the explosion in data volume, formats and requirements there has been a second, quieter evolution in automation. Automation is no longer just a way to generate quick bits of code. It’s become an essential tool in the IT team’s arsenal to face the ever-changing data landscape, answer an increasing business urgency for analytics and adapt faster than the competition.
As the data landscape continues to increase in complexity and business needs continue to escalate and evolve, automation further solidifies its role as an essential component within data infrastructure strategy.
At the Gartner Summit, it was great to see the data community emerging from a time when we needed to evangelize automation and its incredible potential, to one where its benefits are already showing through in multiple data infrastructure automation projects. In this new era, automation can underpin the full lifecycle of data management, from design and development through to the DevOps aspects of deployment, operation and documentation.
Using WhereScape automation software, we’ve seen customers dramatically speed up their design and implementation efforts for data warehouses, delivering faster time to value for businesses even with changing data sources, increased business needs and new types of data such as streaming data. Even more critically, as regulation around data usage increases, WhereScape automation can ensure that supporting metadata and other documentation is kept accurate and up-to-date.
It’s exciting to think of where we will be even a further two years down the line with the adoption of automation. One thing is clear: to survive and thrive in the Information Era, automation is the essential bedrock of ongoing data infrastructure projects in a world where data volume, veracity and velocity continue to explode and the business appetite to more fully leverage data, perhaps a company’s greatest asset, knows no bounds.