Hadoop Summit showcases new generation of big data tools

This week, thousands of IT professionals from around the world are descending on San Jose in California for the 2015 Hadoop Summit North America in order to learn more about the latest developments for the platform. And several providers have been using the occasion to unveil new technologies that will help make the big data tool faster, more secure and easier to use.

Taking place between June 9th and 11th, the event features over 160 sessions, with more than 4,000 developers, data scientists, enterprise architects and solution providers taking part in community activities and listening to keynotes from the likes of Hortonworks, Forrester Research and Yahoo!.

Among the speakers on the first day was principal analyst at Forrester Research Mike Gualtieri, whose address focused on what the future will hold for Hadoop. He noted that for many scenarios, the technology will become the only option for big data deployments as it disrupts the economics of the data analytics industry.

He said: "It's a data operating system and a fundamental data platform that in the next couple of years 100 per cent of large companies will adopt," eWeek reports.

Mr Gualtieri was not the only speaker at the event to predict a bright future for the technology. For instance, chief information officer at GE Software Vince Campisi said the solutions will help break down data silos and help unlock a new 'industrial internet', where sensor data and predictive analytics are used to help manufacturers anticipate when, for example, components are likely to break. This means they can replace them before problems emerge – saving millions of dollars in downtime.

Traditional data warehousing and analytics tools will be unable to cope with the expected 50 billion Internet of Things devices that will be sending information back to companies in the coming years, Mr Campisi said. But Hadoop is able to scale to meet these immense data volumes, offering data scientists insights into relationships "we didn't even know mattered".

Meanwhile, the Summit also saw several major Hadoop players unveil new updates. Among the most prominent of these was MapR Technologies, which introduced version 5.0 of its MapR Distribution.

Central to the announcement was improved support for real-time and multi-tenancy applications, which reflected the demands of today's big data users. Chief marketing officer at the company Jack Norris observed nearly one in five of MapR's customers have 50 or more applications running on a single Hadoop cluster.

"That means you need to support workload management and multi-tenancy," he said.

Elsewhere, Hortonworks unveiled its Hortonworks Data Platform 2.3, which aims to eliminate some of the complexities that are currently a barrier to adoption for many enterprises.

Some experts have raised questions recently about whether Hadoop will be able to live up to its potential, after a survey by Gartner suggested that only 18 per cent of businesses have plans to invest in the technology in the next two years, despite promising results from early adopters.

However, Mr Gualtieri said that in an era where businesses are finding it difficult to get to know their customers, Hadoop will provide the type of personalised service that consumers expect.

"[Hadoop] can give businesses an edge," he said. "You can predict what’s best to put in your online catalog; sensor data in the car ahead of you will alert you when there’s slippage ahead; and you’ll see all the presidential candidates heavily use predictive analytics like Obama did in his first two campaigns to identify swing voters."