Three-quarters of IT pros say big data must improve

Many firms may need to think more carefully about how they incorporate big data analytics into their operations and derive insight, as a large percentage of IT professionals have admitted to disappointment with the initial outcomes of their investment.

This is according to a new survey by Actian, which revealed that even though 61 per cent of respondents rated data as their company's most valuable asset, 77 per cent reported their current analytics deployments have failed to live up to expectations.

However, these early setbacks do not appear to be putting businesses off, and there remains widespread recognition of the potential of big data analytics if it can be leveraged effectively.

More than four-fifths of respondents (81 per cent), who included C-level executives, IT leaders and data scientists spanning more than 25 industries, said that using data analytics to drive business growth is their number one priority for the next 12 months. This was followed by gaining customer knowledge and insight, which was cited by 58 per cent.

Ashish Gupta, chief marketing officer and senior vice president of business development at Actian, said that the current environment is a very disruptive time for IT departments as organisations seek to use data to get ahead of competitors and create new revenue and customer pathways.

"Painful trial and error has revealed that traditional database technologies are failing to deliver on analytical workloads, so they have turned to Hadoop for help," he continued. However, Mr Gupta observed that while Hadoop is a very cost-effective way of storing massive amounts of data, many users say it is not yet mature enough to handle the enterprise-grade, high-performance analytics jobs businesses demand.

More than half of those surveyed (51 per cent) agreed that Hadoop has the potential to make their existing data analytics operations more effective, but only five per cent of IT professionals questioned are pushing for the technology.

One reason for this may be the significant skills gap, which continues to be a barrier to adoption. Although a third of respondents said that Hadoop can provide a cost-effective, scalable way of storing very large data sets, 20 per cent also said that making the most of the technology will require talent they do not possess, while the same percentage said tools are needed to make it more enterprise-grade, secure and fast.

"As with any technology advancement, Hadoop has areas for improvement," Mr Gupta continued. "Many organisations have invested time and resources into making it work for them because they know that their current way of managing analytical workloads won't cut it."

Because many IT professionals are concerned about the potential consequences of a failed big data deployment, they would prefer to stick to tried and tested tools. When asked what they would change about Hadoop, the top response was to make it easier to access data via SQL and traditional business intelligence solutions. This was named by 40 per cent of respondents.