What will Hadoop look like in 2020?

Over the last couple of years, Hadoop has emerged to be one of the leading technologies for any firm that's looking to get involved with big data. The open-source platform promises to offer affordable, powerful solutions for the management and processing of large data sets, so it's no wonder interest is booming.

It was noted by eWeek that although Hadoop is still used sparingly in many enterprises at the moment, this is set to change in the coming years as recognition grows of just how useful it can be throughout all aspects of a business.

"In the next five years, more enterprises will adopt a data-driven business and will look to Hadoop to support their growth," the publication said, adding that Hadoop will allow for "leaner, faster and more efficient enterprises".

This focus will create new challenges, as well as opportunities for new entrants into the Hadoop market to find a place. As eWeek highlighted recent figures from Allied Market Research that forecast the Hadoop market will be worth $50.2 billion by 2020, there will be huge incentives for providers to offer new solutions for Hadoop.

So what will the Hadoop landscape look like by the end of the decade? eWeek highlighted several factors that will become apparent over the coming years that firms will have to be aware of.

For starters, the site noted that Hadoop will lead the way in infrastructure spending by 2020, when most enterprises will have strategies in place to leverage the technology. This will make Hadoop the greatest infrastructure investment for many companies.

However, despite this, the technology will not be the greatest expense IT professionals have to face. It was noted this is not due to a lack of interest, but because the technology is only around one-tenth of the cost of legacy data warehouse solutions. This means resources will be freed up for use elsewhere.

Hadoop will also be used for data storage and processing across all parts of the enterprise. It was stated by eWeek that a recent survey has found 13 per cent of respondents are already using Hadoop in production or pilots – already surpassing the ten per cent of data that some industry experts have claimed Hadoop will affect. This upswing is set to continue over the coming years, spreading Hadoop throughout businesses.

Use cases for Hadoop are also expected to expand dramatically over the coming years, with many day-to-day operations set to adopt the tools.

It was stated by eWeek: "As Hadoop is used more and the capabilities of YARN become fully realised, more useful opportunities leveraging technology like Apache Spark and Storm will emerge and quickly increase its potential. Even now, real-time/operational analytics are the fastest moving part of the Hadoop ecosystem, and by 2020, Hadoop will be relied on for day-to-day enterprise operations."

The technology will also help to 'democratise' data. The publication suggested the low costs of Hadoop will enable more companies to create and manage their own data warehouses. So far, the inherent complexity of the technology has acted as a barrier to adoption, but given the current pace of innovation and low cost of entry, it was estimated that small and medium-sized enterprises will be investing in their own Hadoop-based infrastructure within five years.