Companies gathering and analysing large amounts of potentially sensitive customer data have been urged to follow best practices that can mitigate the risk of failures and potential ethics violations.

According to Gartner, failing to properly appreciate the dangers can have a number of significant consequences if big data projects fail to deliver the expected results. Alexander Linden, research director at the organisation, said that while advanced analytics projects come with many of the same pitfalls as more traditional projects, their risks are accentuated by the high volume and variety of data involved.

As a result of this, Gartner forecast that by 2018, half of all business ethics violations will be the result of improper use of big data analytics. This can lead directly to issues such as loss of reputation, limitations in business operations, losing out to competitors, inefficient or wasted use of resources, and even legal sanctions. 

Therefore, the organisation highlighted several key best practices that businesses should be following in order to minimise the risk of such big data failures.

For instance, Gartner noted that an effective analytics solution must help business decision-makers take action – and these actions need to have measurable effects in order to demonstrate the success of the programme. 

"Linking analytic outputs to traceable outcomes using a formal benefits-management and mapping process can help the analytics team navigate the complexities of the business environment, and keep analytic efforts both relevant and justifiable," the research firm stated. 

It also observed that businesses should balance the analytics processes they engage in with their ability to actually make use of the results they receive. Not every scenario will lend itself to an analytics based approach, so enterprises need to be wary of turning automatically to such solutions before they have fully assessed their situation.

For example, if the most relevant data is missing, or the available information is highly ambiguous, effective big data solutions may be difficult to implement. This means businesses may open themselves up to all the risks that come with this for very little chance of reward.

Taking an incremental approach to analytics was also highly recommended by Gartner. Instead of immediately jumping in with initiatives that aim to transform the way businesses operate, a more measured approach that seeks to improve existing analyses or update and extend an existing business process will be easier to achieve – and come with fewer consequences should things go wrong.

Enterprises should also remember that there will usually be more than one route to their goal, so they need to consider alternative approaches if their initial ideas prove to be too high-risk.

"Statistical modelling, data mining and machine learning algorithms all provide means of testing ideas and refining solution propositions," Gartner said. "Big data and advanced analytics help validate proposed hypotheses and open an even wider range of potential approaches to addressing corporate priorities."