Why Data Management needs a serious rethink

Matthew Johnston, Area Vice President, ASEAN & Korea at Commvault

We are running out of space—fast! Digital transformation, Big Data analytics, new regulations on data protection and security, and data-driven business strategies are fueling data growth to stellar proportions. In turn, it has made storage space a premium.

To avoid the data space crunch, firms are buying capacity in droves. IDC noted that total capacity shipments were up 18.3% year over year to 52.4 Exabytes in Q4 2016 alone. Marketsandmarkets predicted that cloud storage, a benefactor of cloud adoption, will grow at a compound annual rate growth rate (CAGR) of 25.8% from 2016 to 2021, becoming a US$74.94 billion dollar market. And when Internet of Things (IoT) comes online, the data storage explosion will implode unprepared data infrastructures.

However, throwing more space at data growth does not solve the problem. Rather it is only addressing the symptom. Instead, firms need to rethink their data management strategy holistically if they want to avoid a data implosion.  

Restarting Data Management

In the past, most storage administrators looked to control data volumes. Different storage architectures were used to rein in storage growth.

As today’s companies embrace digital transformation and increasingly rely on data to be efficient, data volumes are growing exponentially faster than ever before. A big contributor to this unrelenting growth: unstructured data, i.e. emails, videos, documents, etc. Many previous storage infrastructures were not specifically designed to handle this data type efficiently.

But managing data volumes is only one aspect of data management. Companies need to also handle variety and velocity of data as they shift to a digital-centric and mobile-driven environment.

Matthew Johnston, Area Vice President, ASEAN & Korea at Commvault“Data management is a broad term. But what we are really talking about is the value of data. The reason we manage data is because it has value. And to understand the value you need to categorize it,” said Matthew Johnston (photo right), Area Vice President, ASEAN & Korea at Commvault, adding that data lifecycle management is crucial.

See the big picture

The cloud can offer much needed relief. But without a strong data management strategy, it is not going to solve all your data management problems. Instead, Johnston believes firms should take a step back and start examining the data itself.

“First step is to understand your data intimately and its relevance to the business. All data are not created equal. Some data is more critical. So, you need to understand what can be placed in the cloud,” said Johnston.

A clear understanding of the data and its value to the company can help firms to understand where and how to store it. It also allows the firms to look at new ways to mitigate data-related risks in case the unthinkable occurs—and it does.

Consider the latest Amazon Simple Storage Service (S3) issue. A simple debug of a billing system took down the servers. The issue disrupted websites and businesses. If you are an e-commerce company, it may mean millions in lost revenues. 

Meanwhile, Gitlab, a well-funded and known startup that saw itself as an alternative to GitHub, recently suffered a similar outage when an employee deleted the wrong files. Although the mistake was unintentional, the damage was done. What was worse is that they had to inform their customers that data created within the past six hours were lost forever.

“All [cloud service providers] are offering is the infrastructure. The responsibility to look after the data is yours,” said Johnston, adding that a sound data management strategy should be a firm-wide concern.

Johnston highlighted four key steps that firms need to consider when architecting their data management strategy: manage data by region; clearly know where your data lives; make sure you have an alternative data recovery plan; and have a holistic data strategy.

“Once companies understand the data and have the [right policies], it is then about choosing the right technology that gives them the flexibility to move the data,” said Johnston.

SDN to the rescue

Johnston urged companies to stop looking at data as a storage problem. He noted that companies need to move away from this short-sighted view when building their data management strategy—especially when considering the complexities of operating in a digital world.

“Take IoT for example. Knowing the location of your devices is only information at a certain point in time. To make better sense of the data, you need to store and correlate it, which can mean a huge volume over a period time. Also, you need to understand how you can protect and store it for longer periods to draw long-term correlations,” said Johnston.

He advised companies to take a hard look at Software-Defined Data Services (SDDS). It essentially abstracts the data away from the storage array, allowing complete portability of data across platforms whilst maintaining the same user and application experience. This expands the value of the data. It also means companies can create a virtual data lake without having to move all the data into a single repository.

“From our perspective, we become essentially [hardware vendor] agnostic. We support data regardless of where they are stored and in which format. It also does not change the policies on how you manage your data. What it gives you is agility from a storage perspective,” said Johnston.


Simon Robinson, Research Vice President at 451 and Research Director of the new Voice of the Enterprise: Storage service, highlighted that the storage infrastructure is not immune to the uber-trends fundamentally disrupting the role of IT at organizations.

He noted that IT managers need to recognize the need for storage transformation to meet the realities of the digital economy, particularly around the areas of efficiency and agility. He added that public cloud storage and all-flash array technologies will be key components of this transformation.

Johnston believes that a solid data management strategy can help firms to successfully transform their storage infrastructure. More importantly, it will help firms to understand how data directly and indirectly impacts the business without being distracted by new storage paradigms or infrastructure solutions.

“Then we can apply data management strategies. It is not rocket science but a reflection of your business strategy and processes,” he said.

The good news is that firms are beginning to consider data management strategies seriously, highlighted Johnston.

“Organizations these days are becoming smarter at how they view information technology. We are now seeing CIOs becoming part of the business. And as data becomes an important currency, data management becomes key to the business strategy,” he concluded.