‘Fast data’ the next stage for analytics deployments

Businesses need to be increasingly thinking about the speed of their big data analytics operations, as well as their size, if they are to make the most of the technology available to them, it has been stated.

It was noted by Tech Republic that that as recently as last year, the focus for many initiatives was primarily on gathering as much information as possible to feed into analytics solutions. However, this approach does not take into consideration the two other key pillars of the technology, variety and velocity, and it is the latter that is now of particular interest.

Kevin Webber, developer advocate for Typesafe, told the publication that modern software is increasingly required to act on data in real-time, as soon as it enters a business, rather than simply analysing it at rest at a later time. 

This will result in enterprises needing to develop a new approach for their analytics, moving away from traditional batch processing to a stream-based architecture.

"In these systems, live data is captured, processed, and used to modify behaviour with response times of seconds or less," Mr Webber said. "There is major business value in sub-second response times to changing information."

He added that ten years ago, the speed of many operations was dictated by the technological limitations of the time. In many cases, critical pieces of market information would not become available to companies until the day after events occurred, by which time it would be too late to act upon it.

Now, however, the ability to get this essential information as soon as it happens will create a lot of value for businesses.

"Fast data is critical to fast knowledge, and businesses want knowledge as quickly as possible," Mr Webber said.

However, businesses will need to put the right tools in place to cope with this. For instance, they will require solutions that prevent their systems from becoming overwhelmed by the incoming data.

"If you think of real-time data as water flowing down a river, at some point, heavy rain may cause the water to overflow and flood the surrounding area," Mr Webber explained. Therefore, systems need a mechanism to provide resilience, by ensuring that all participants in a stream-based system use flow control for a steady state of operation.