Even though big data is now past the hype phase and an established technology forRead More
Many companies ‘still struggling with poor-quality data’
Even though big data is now past the hype phase and an established technology for many businesses, a large number of organisations are still struggling to make the most of their tools. One of the most common reasons for this is that the information they are working with is not good enough for effective analysis.
It was noted by eWeek that if data is poor quality or not managed properly, no specialist big data tools will be able to provide accurate results. In particular, scattered enterprise data environments mean analytics teams are having difficulty accessing and analysing data in a timely manner.
This is especially problematic when it comes to dealing with unstructured data, which is often harder to sort and store in a logical manner. Enterprises therefore need be proactive when it comes to unlocking the value of unstructured 'people data', in order to enhance business decision-making and performance.
Kon Leong, chief executive and co-founder of ZL Technologies, also told the publication that "the mainstream analytics paradigm is broken". He added: "Tedious sampling and use of point tools undermine big data's potential."
Therefore, eWeek highlighted several common data problems that businesses need to be aware of. For instance, it warned that 'dirty' data can cause major headaches for businesses, as it can make analysis difficult and skew the results. This can include duplicate copies, corrupted files, missing sets of information and data that has been held beyond appropriate retention periods.
Outdated information is another frequently-encountered issue. While most organisations have policies in place to delete data once it has reached the end of its useful life, these are not always followed in practice, which leads to irrelevant information being mixed in with more timely details.
Businesses must also work hard to overcome any inherent biases that can affect the quality of their data. It was noted by eWeek: "All the analytics in the world won't fix data that was gathered in a biased manner or data that is naturally skewed due to systematic business practices."
When it comes to analytics operations themselves, organisations need to be aware of both what is being examined and who is doing it.
For instance, in industries such as finance or healthcare, regulators will not be interesting in who is looking at the data – if any red flags are found, the company as a whole is assumed to have been aware of any issues. Therefore, if a pharmaceutical marketing team analyses public social posts, for example, but finds mass complaints about adverse effects, they need to immediately escalate those findings to internal risk guardians who are in a positions to take actions.
Even if analytics is performed effectively, companies need to be prepared for the possibility that the results they receive may not be what they want. It is all-too-easy for companies to overlook negative data, such as trends of employee harassment, compliance violations and public outcry, in favour of results that are more positive for the business.
However, such findings must be addressed quickly, as eWeek noted practically all enterprise data is potentially discoverable in litigation, and problematic analytics findings can later harm the business if they have not been acted on.