Key mistakes to avoid when adopting big data

With recognition of the importance of big data growing all the time, more businesses than ever are looking to get involved with the technology and implement their own solutions.

However, this can be a process that's fraught with danger for the unwary company. Jumping straight in without careful planning or a clear understanding of what they want to achieve from the initiative can quickly lead to disappointment and dismissing big data as overhyped.

And some of the most common mistakes businesses make when deploying big data solutions have been highlighted recently by Jon Bakke, executive vice president of worldwide field operations at NoSQL database company MarkLogic. In a piece for VentureBeat, he noted that these errors often do not result from the technology itself, but a poor understanding of how to incorporate it into a business.

For instance, not doing enough with the data firms gather is one of the most common flaws in a strategy. There can be a number of reasons for this, such as failing to recognise the potential of certain information beyond its most obvious uses, or not upgrading solutions such as databases to fast, modern tools that can process the information quickly enough.

Related to this is a lack of knowledge about exactly what details are available for analysis, and what companies might get out of this. Mr Bakke noted that many firms are currently struggling with a 'knowledge gap', where staff do not know how to meet the specific challenges facing their organisation.

"Many have scoffed at the need or cost of bringing new experts into organisations, but organisations without appropriate expertise struggle to understand their own data, what it all means, and the best way to use it," he said.

This does not necessarily mean businesses will have to invest heavily in employing a dedicated chief data officer of team of data scientists, however. Mr Bakke said that in some cases, having the right big data analytics platform and support can help existing staff gain the tools and expertise needed to interpret information.

He also advised companies to think carefully about their current capabilities when planning their big data initiatives, as one of the most common reason for failure is firms biting off more than they can chew.

"There is a failed notion that all big data issues have to somehow be solved together like one big monolithic problem requiring a single monolithic solution," he said. "Leading with the end game in mind, IT managers and chief information officers should be asking what business decision they're trying to affect, rather than how to integrate new technology into existing technology." 

By starting small and scaling up quickly once teams are comfortable with the technology, firms will be better able to keep projects on time, on budget and yielding the desired results.

Mr Bakke noted that big data now enompasses all industries, whether it is financial information, healthcare records, retail analytics, government intelligence. The only consistency within this is that the volume, variety and complexity of data will continue to grow, so organisations need to understand the potential pitfalls for why big data projects fail, and plan carefully to avoid these issues.