Use of Hadoop exploding, creator says

The number of businesses that are recognizing the benefits of using the Hadoop platform is "exploding" as firms from across the economy seek better ways of managing their growing information volumes.

This is according to creator of the solution Doug Cutting, who told ZDNet that it has become clear in the past few years that the industry has accepted the platform as the standard solution.

He stated: "There's no question it's exploding across every industry. Every industry has lots of data and they want to be able to store it and process it economically."

Mr Cutting observed the range of potential use cases for Hadoop has greatly expanded recently due to developments that have added capabilities such as the ability to conduct real-time SQL queries and a Google-like keyword search for tracking down information.

"I think SQL and search are the two primary search methods that people have for querying data, so we've got those covered," he said.

The additions have greatly improved the performance of Hadoop, Mr Cutting said, which should make it easier for businesses to conduct in-depth data analytics on large sets of information. 
He noted the Impala engine – which allows SQL queries to be run against the Hadoop Distributed File System or the non-relational HBase database – enables results to be returned in real-time, up to ten time faster than older query engines.

The ability to carry out Google-like keyword searches across millions of documents in a Hadoop cluster also allows relevant information to be brought up in "a fraction of a second", Mr cutting continued.

Speaking about the future of the technology, he added a key priority will be to add improved security and auditing tools to ensure companies across all industries – including those with strict regulatory requirements – can benefit from Hadoop. These will be particularly beneficial to sectors such as finance, government and healthcare, it was stated.

Other additions said to be in the works include finer-grained authorization controls. Mr Cutting observed: "Within a HBase table you can now control the access to different rows and columns, eventually we'd like to get cell-level authorisation in HBase."