Skills gaps ‘the key barrier’ to Hadoop adoption

A lack of skills and uncertainty over how to gain value from Hadoop are among the main reasons many businesses are still avoiding the technology when looking for big data analytics solutions.

This is according to a new survey from Gartner, which revealed more than half of businesses (54 per cent) currently have no plans to invest in Hadoop, while only 18 per cent expect to adopt the technology within the next two years.

Only a quarter of respondents (26 per cent) say they are either deploying, piloting or experimenting with the technology at the moment. When asked why they were not seeking to invest, several IT professionals stated that it is not a priority, while some companies believed Hadoop is 'overkill' for the problems they face, which Gartner said implies there are worries that the costs of implementing it will be too high relative to the expected benefit.

A key issue was a skill gap within organisations, with 57 per cent of respondents identifying this as a barrier to Hadoop adoption. Meanwhile, almost half of professionals (49 per cent) stated they were having difficulty figuring out how to derive value from the platform.

Gartner observed that the absence of skills has long been a blocker and, while tools are improving to address this issue, today's solutions primarily support highly-skilled users. The research firm estimated it would be two to three years before this challenge is fully addressed. 

Among the firms that have adopted Hadoop solutions, the scale of rollouts remains limited. Some 70 per cent of respondents who are actively piloting or deploying Hadoop report small numbers of users accessing the cluster – having between one and 20 users accessing Hadoop. Surprisingly, four per cent of these businesses reported zero users. 

Nick Heudecker, research director at Gartner, said: "Early Hadoop projects typically involve a small number of users and this no doubt keeps user populations down at this stage of the market.

"Moreover, the Hadoop stack remains unsuitable for simultaneous use by multiple users, also keeping numbers down. Another factor, and possibly the best explanation for the small number of Hadoop users, is the skills shortage."

He added that one of the core value propositions of Hadoop for many users is that it offers a lower-cost option to traditional information infrastructure. However, the low numbers of users relative to the cost of cluster hardware, as well as any software support costs, may mean Hadoop is failing to live up to this promise at the moment. 

Therefore, businesses looking to the technology will need to have a clear plan in place to identify how they will gain a return on investment from it.