The number of businesses turning to Hadoop to assist with their big data analytics operationsRead More
What will 2016 hold for the big data industry?
The last 12 months have been a busy time for the big data industry, as more businesses have begun to recognise the value of the technology, while many initiatives are starting to move out of proof-of-concept tests and into full production.
But as 2015 comes to a close, many professionals will naturally be turning their attentions to the future and wondering what it will hold for the technology. And with the maturity of big data growing rapidly and more best practices emerging, there are major changes ahead.
For instance, it was forecast by Oracle that in 2016, one key trend will be that big data is no longer limited to the experts. In a piece for the Predictive Analytics Times, vice-president of big data and advanced analytics at the company Neil Mendelson and the firm's vice-president of big data integration and governance Jeff Pollock observed that "data civilians" will find themselves operating much more like data scientists.
"While complex statistics may still be limited to data scientists, data-driven decision-making shouldn't be," the pair stated. "In the coming year, simpler big data discovery tools will let business analysts shop for datasets in enterprise Hadoop clusters, reshape them into new mashup combinations, and even analyse them with exploratory machine learning techniques."
Providing a wider audience with the tools to explore their data will help businesses improve self-service capabilities and enable more users to develop hypotheses and experiments based on the insights provided by big data.
As a result of this, experimental data labs are also set to take off in 2016. Mr Mendelson and Mr Pollock said, for example, that the insurance sector will trial a wide range of tests and pilot schemes related to default risk, policy underwriting, and fraud detection as firms try to identify the best way of utilising their data before their competition.
Meanwhile, the emergence of more mature technologies and best practices will mean many businesses no longer have to go it alone and create DIY solutions for their big data analytics.
"In 2016, we'll see technologies mature and become more mainstream thanks to cloud services and appliances with pre-configured automation and standardisation," the experts stated. This will mean that lead-in and development times for big data tools will fall considerably, as well as making deployment significantly cheaper.
They also forecast that in 2016, the emergence of more Internet of Things (IoT) enabled sensors will join forces with powerful cloud computing tools to become the 'killer app' for big data analytics. Expanding cloud services will not only be able to gather sensor data, but also feed it into big data analytics and algorithms to turn it into actionable results.
Highly secure IoT cloud services will also help manufacturers create new products that are able to act upon the analysed data without the need for human intervention.
However, in order to ensure that these innovations can be used safely and effectively, issues surrounding data governance and security will also come to the fore in the next 12 months.
"The continuous threat of ever more sophisticated hackers will prompt companies to both tighten security, as well as audit access and use of data," Mr Pollock and Mr Mendelson said. They added it will also be increasingly important to know where data originates – not just in terms of what sensor or system generates it, but which country's jurisdiction it will fall under for data protection and privacy purposes.