The ability to dramatically cut storage costs and create a solution that can be easily scaled up on demand will be the driving force behind many companies' decisions to implement Hadoop in the coming years.

This is according to chief executive of WANdisco David Richards, who told ComputerworldUK that while many businesses are currently trialling the technology to help them gain insight into unstructured data, the economic benefits of Hadoop will be the catalyst for more widespread mainstream adoption.

In many cases, businesses will be forced to look towards Hadoop to make sure they do not fall behind competitors.

"Imagine a world in which one bank has Hadoop and everyone else is using proprietary storage – it is not going to work," he explained. "One bank will have a massive competitive advantage because they can scale up storage at ten per cent the cost of everybody else." 

He added that requirements for data storage are currently climbing at a compound annual growth rate of around 60 per cent, as big data begins to play a more central role in many companies' operations and the amount of information available to be collected soars. But budgets remain mostly static, so proprietary storage solutions will not be a practical solution for meeting this demand for data. 

Mr Richards also noted that the Hadoop market is set for a significant step forward this year, as many adopters of the technology move beyond their evaluation and trial stage and begin to run applications live in production.

In many cases, the 'lab period' for Hadoop has lasted for between one and two years, and this is now coming to an end for many early adopters who were keen to jump on the Hadoop bandwagon and experiment to see what it could offer to their business.

Although there are many companies that report they are using Hadoop at the moment, many of these will not yet be in a situation where the technology is able to have a significant impact on their bottom line, as it is yet to be incorporated fully into their day-to-day operations.

“You see a lot of companies say they are using Hadoop, and my first question is 'what is the SLA'. If that application goes down for two hours, what happens?" Mr Richards said. "If they say 'nothing' then you know it is in a lab, it is not being used for a strategic business reason."

However, this is beginning to change as more businesses reach the end of these trial periods. Mr Richard observed: "I think there are so many companies – healthcare companies and so forth – that are coming out of the lab, that are using this for strategic purposes, and we are about to see this market evolve very quickly."

For instance, all of the Forbes Global 2000 companies have some form of big data strategy in place, and these trials will come to fruition over the next year or so.