News and events

In-memory technology ‘to be driving force’ of future big data

With companies having to deal with more information than ever and facing increasing demands for real-time results, in-memory technologies are set to play a major role in data analytics in the coming years.

This is according to a new report from the Bloor Group, which forecast that enterprises will have to rely more heavily on this solution in the coming years if they are to make the most of the opportunities afforded to them by the new era of business intelligence (BI).

The report, titled Why In-Memory Technology Will Dominate Big Data: In-Memory and the New BI stated that the faster time to insight offered by this solution could be one of the key differentiators between the most successful firms and their rivals. Failure to adopt such tools could hamper the results companies get from their big data analytics deployments.

Author of the report and principal analyst at the Bloor Group Robin Bloor PhD said: "If data analysts can get results in seconds or minutes instead of hours, then they can test many more hypotheses than before – and they will. They are also more likely to consider engaging in activities they previously thought impractical, such as adding to their data sources and pursuing new analytical projects."

This use of advanced and predictive analytics helps data scientists reveal greater insight and take immediate advantage of developing trends. Stock transactions were cited as one area that can especially benefits from the faster results offered by in-memory technology, along with transportation, telecommunications, retail, media and web-based businesses.

Dr Bloor also highlighted the Kognitio Analytical Platform as one example of how in-memory technology is being applied successfully. He observed that this solution optimizes all data brought into memory to be accessed on a truly random basis, instead of the block-basis query solutions used by traditional database tools. In addition, it was noted that being able to read from memory is as much as 3,300 times faster than reading from disk, which means calculation times can be cut from hours to just seconds. 

In the past, analytical tasks were typically a three-step process, he explained. The solutions would have to read memory from disk into cache, answer the direct direct and then apply the analytical processes to produce a final result. With in-memory tools such as the Kognitio Analytical Platform, however, this is reduced to a single step as the query and analytics are run in parallel.

As well as running standard SQL queries, Kognitio's offering also "can also execute analytical processing, running third-party binaries embedded directly within SQL commands," Dr Bloor said.

He added that using in-memory platforms such as that provided by Kognitio can give companies a major strategic advantage over less-advanced competitors. Understanding how best to use this technology will help businesses spot trends early instead of reacting to past data points. "If BI dashboards can be accurate up-to-the-second, then it may be possible to support individual decision making in the moment, rather than report 'after the fact'," he stated.