Opportunities for Hadoop ‘never bigger’, says Hortonworks boss

The potential for Hadoop-based big data analytics systems to revolutionise how companies handle their data has never been bigger, so IT professionals need to act in order to take advantage of this.

This was the message of Hortonworks chief executive Rob Bearden, who was delivering a keynote address at last week's Hadoop Summit in San Jose, California, Datanami reports. He laid out an optimistic future for the technology platform, observing: "We may never have an opportunity this big again in our careers."

Although some analysts have raised concerns recently about the slow pace of Hadoop adoption – most notably Gartner's Nick Heudecker, who described the forecast for the next two years as "fairly anaemic" – Mr Bearden is of the opinion that the adoption rate is where it should be at this stage in the technology's development.

"If you draw a parallel to where the relational database was five years into its formation, which at this point was 25 to 30 years ago, I think arguably Hadoop is on a much faster adoption and ramp curve," he said.

He also highlighted a study by market research firm ETR that revealed Hadoop has the highest net spending intention of any of the 25 enterprise IT categories it tracked for 2015. In other words, companies plan to spend more on Hadoop than on any other major technology, such as information security, social media and cloud computing.

One of the key reasons for this is because traditional relational databases cannot handle the huge volume and variety of data that has emerged in recent years. Mr Bearden explained that a new wave of data sources has impacted on the enterprise in the last three to four years, and this has put pressure on traditional data architecture and transaction systems, which have become very siloed and constrained.

However, moving to Hadoop does not mean that these old systems will have to be abandoned completely. Mr Bearden said: "None of the existing capabilities go away. I think that was always what was so powerful when we started looking at the Hadoop opportunity. It didn't have to be a zero-sum game for Hadoop to win."

The scalability of Hadoop will also help create opportunities for users across every industry. The expert said that there will be "tremendous value" in bringing all of a company's data together under a centralised management solution. This modern data architecture will open up use cases and business models that were simply not possible in the past.

Mr Bearden also observed that the opportunity that Hadoop has to transform the big data analytics environment has only come about due to its open source nature, which allows the community to collaborate on development and come up with innovative new uses for the solution.

"I've said from day one that Hadoop is too big of a platform and too big of an opportunity for any one company," he said. "Hadoop is about a community … I think we should be very proud, especially in the last three or four years, of what we've accomplished together as a team and as a family."