Spotting the signs your big data project is heading for failure

big data project, buisness departments, IT

The number of companies embarking on big data projects is set to increase hugely in the coming years, as awareness of the technology grows and more enterprises come to understand what it can offer them.

But making such deployments work effectively is no easy task, and the size of the business and the amount of resources it is able to devote to such projects may have little bearing on its success. According to Gartner, 85 per cent of Fortune 500 organisations will be unable to gain competitive advantage from big data in 2015, and only 60 per cent of initiatives will ever make it to the production stage.

There are many reasons why this may be the case. But often, there will be a few key warning signs companies should be alert to that may indicate their projects are heading for failure. By learning how to spot these signs early, businesses can change course and put themselves on the right path to ensure a positive outcome.

Who’s leading your efforts?

An effective big data analytics initiative needs to have input and engagement from all departments, including the IT team tasked with the implementation, the business units that will actually use it, and the board members who sign off on it. However, in many cases, organisations end up leaving the project in the hands of one team, which can create a range of problems.

For instance, some firms may consider it a good idea to allow their marketing teams to take the lead, as improving this department’s performance is a common goal for big data. However, without input from IT, these efforts may be difficult to monetise, or transfer across to other departments as users fail to fully understand how to utilise the technology.

On the other hand, it may seem to make sense to hand the reins over to data warehousing teams, as they will typically have vast experience with traditional business intelligence solutions. However, the consequence of this is they will likely be resistant to the type of wholesale change necessary to make big data work and, if left to their own devices, will not deliver a system fit for today’s purposes.

What’s your plan for progress?

Setting out a clear roadmap for the development of a big data tool is essential, but if key steps are done too early or too late, this can seriously hinder the effectiveness of a solution. For example, one common assumption is for a business to decide that they will need a skilled data scientist to manage their solution.

Therefore, they turn to sources such as LinkedIn seeking individuals with this skill listed, before they fully understand what their requirements are or what skills they will need. This can leave them unable to determine whether the candidates they are considering are really suitable – or if they have personnel already in their organisation that can be trained.

Similarly, it can be tempting to settle quickly on a technology to base a big data analytics platform on. Many firms are keen to get this done quickly, as it feels like once a solution has been selected, the bulk of the work is complete. However, once a tool is selected, a business is locked into a particular path, which can be time-consuming and costly to change if it later emerges the platform isn’t suitable.

Do you have the right expectations?

Another important warning sign that a big data analytics project may be likely to fail is if personnel – particularly at the board level – are taking a too results-driven approach to the initiative. This will be especially noticeable if advocates for the solution are being quizzed about expected ROI right from the start.

One of the problems with this is that it is usually difficult to put a clear monetary value on a big data project. It is often an unpredictable process of trial and error, and trying to consider returns via traditional methods may be doomed to failure, and inevitably lead to disappointment.

On the other hand, expecting to wait six months or more before seeing any results is also a sign of poor planning. Given the fluid nature of big data, it’s unavoidable there will be some failures along the way – the secret to success is to start small, ‘fail fast’ and respond to these failures as quickly as possible. If businesses are not planning on looking at results until six months down the line, by the time they identify potentially fatal problems, it will be too late to do anything other than start from scratch.