The vast majority of companies in the retail, technology, banking, healthcare and life sciences sectors will be investing in real-time analytics tools for studying human and machine-generated data.

According to research conducted by OpsClarity, 92 per cent of these organisations expect to increase their focus on streaming data applications within the next 12 months.

To do this, almost four-fifths of respondents (79 per cent) will be reducing investment in batch processing tools, or even eliminating these entirely, as they shift their resources to real-time analytics.

Dhruv Jain, chief executive and co-founder of OpsClarity, stated that the ability to study data in real-time can give businesses a significant competitive advantage in today's digital economy, allowing them to become more agile and innovative.

"With new fast data technologies, companies can make real-time decisions about customer intentions and provide instant and highly personalised offers, rather than sending an offline offer in an email a week later," he said. "It also allows companies to almost instantaneously detect fraud and intrusions, rather than waiting to collect all the data and processing it after it is too late."

One of the key use cases for this technology will be to enhance customer-facing applications. OpsClarity noted that businesses are now able to leverage insights gleaned from multiple streams of real-time data in order to enable timely decisions and responses to queries. 

This type of real-time analysis is now being built directly into customer-facing, business-critical applications. A third of survey respondents (32 per cent) said their real-time solutions would be used primarily to powercore customer-facing applications, whereas 29 per cent will be focusing on improving internal processes.

Almost four out of ten professionals (39 per cent) said they would be deploying real-time data analytics for both purposes.

Jay Kreps, chief executive and co-founder of Confluent, added that real-time data and streaming processes are becoming a central part of how modern businesses harness the information available to them.

"For modern companies, data is no longer just powering stale daily reports – it's being baked into an increasingly sophisticated set of applications, from detecting fraud and powering real-time analytics to guiding smarter customer interactions," he continued.

One of the most popular solutions for handling real-time data is Apache Kafka, with 86 per cent of software developers, architects and DevOps professionals using the open-source message broker.

Mr Kreps noted that this provides a real-time platform for thousands of firms, including major companies such as Uber, Netflix and Goldman Sachs.

Meanwhile, Apache Spark is the data processing technology of choice for 70 per cent of businesses, while 54 per cent use HDFS data sink.

Although there are a wide range of data framework being deployed, and strong indications that many of these will be here to stay for the foreseeable future, the survey revealed a strong preference for open source technologies.

Nearly half (47 per cent) of software developers, architects and DevOps professionals say they exclusively use open source, and another 44 per cent use both commercial and open source.