The growth of the big data analytics industry is set to continue growing at pace over the next few years, which will mean even more firms are likely to seek out the latest technologies and expert support partners to help them make the most of the information available to them.
Research by Gartner revealed almost three-quarters of companies (73 per cent) have plans to invest in this area within the next two years. This is up from 64 per cent last year. The research shows many companies are moving off the fence when it comes to big data, with the number of enterprises with no plans to investigate the technology falling from 31 per cent to 24 per cent.
This will mean many firms will have to address the range of challenges associated with big data analytics – and in particular how they deal with the so-called 'three Vs' of the technology – volume, variety and velocity.
Of these, volume is receiving much of the focus, which Gartner noted is understandable as in many cases, companies are gathering much of the same data they have always done, but at much bigger scales. However, this is also perhaps the easiest issue to deal with, by increasing compute and storage capacity.
But data variety, on the other hand, is set to be far more challenging. Gartner noted that getting value from a variety of information sources – including social media feeds, machine and sensor data and free-form text – requires not only increased storage capacity, but also different tools and the skills to use them.
The research firm added this may be one reason why many firms are sticking to traditional data sources when conducting analytics processes. Organisations analysing transactions increased from 70 per cent in 2013 to 79 per cent in 2014, while those looking at log data fell slightly by two per cent.
However, this may be set to change in the coming years, as Gartner found firms are looking at adding a wide range of sources in the future. Nick Heudecker, research director at Gartner, said that every data source received roughly 30 per cent to 40 per cent of responses, including "extremely challenging" data sources like audio and video.
He described this as "overly optimistic" and indicates many companies do not have a coherent plan for how to make the most of their data.
"Picking everything isn't a strategy. It indicates a fear of missing out on an opportunity yet to be defined," Mr Heudecker continued. "Also, there may be a certain amount of hubris at work. If organisations can 'do big data' on transactions and log data, they may assume they can also leverage more challenging data sources as easily."
Without turning to expert providers to support these projects, businesses may therefore find their deployments do not go as well as hoped, which could dissuade them from making further investments or investigating what else they could achieve with the tools.
However, if rollouts are successful, Gartner noted the range of potential uses for bug data is huge, with every industry able to benefit in some way from the technology.
Lisa Kart, research director at the company, said: "Big data can help address a wide range of business problems across many industries and for the third year in our study, both enhancing the customer experience and improving process efficiency are the top areas to address."
She highlighted transportation, healthcare, insurance, media and communications, retail, and banking as among the sectors that stand well-placed to benefit from this.