A large majority of public sector IT professionals in the US agree that effective implementations of big data analytics tools will be essential if their organisations are to identify and defend against threats to their networks.

Research conducted by MeriTalk on behalf of Splunk Inc revealed that on average, cyber threats exist on government networks for 16 days before they are uncovered.

But despite the fact that 86 per cent of respondents agree their cyber security efforts would be improved through the use of big data analytics to spot threats, only 28 per cent of organisations at federal, state and local agencies are fully leveraging this technology at the moment.

The study found that big data is expected to be useful in a variety of ways when it comes to cyber security. For instance, more than six out of ten respondents (61 per cent) agree that the technology can help improve the detection of breaches currently in process.

More than half of professionals (51 per cent) also stated big data would help them monitor streams of information in real-time, while 49 per cent said such solutions would assist them in conducting a thorough root-cause analysis in the aftermath of a security incident.

However, only one in three government organisations are making the use of big data analytics a priority when it comes to planning their defences, despite the wealth of threat information it can offer.

Kevin Davis, area vice-president for public sector at Splunk, commented: "The challenge is managing that data and connecting the dots in real time. That’s how we get immediate insight into threats. Agencies need to detect threats faster and start to predict when and how they will occur."

One obstacle to this is the sheer volume of data that government organisations have to manage. More than two-thirds of professionals (68 per cent) say they are overwhelmed by the amount of security data.

At a federal level, 45 per cent of IT managers cite data volume as the biggest challenge when it comes to fully leveraging big data analytics for cyber security, while 54 per cent of state and local government IT managers cite lack of resources, specifically skilled personnel.

What's more, almost four out of five respondents admitted that at least some of their data goes unanalysed, either due to a lack of time or because their teams do not have the necessary skills and tools to interpret it.

To meet these challenges, government agencies stated they need better funding, training and support from management in order to develop more proactive, big data-based cyber security strategies that fully leverage the available information.

The survey found positive steps are being made towards this. Some 92 per cent of professionals are working to improve cyber security, by investing in new or upgraded security technologies (65 per cent), deploying network analysis and visibility solutions (51 per cent), and improving training (50 per cent).

Steve O'Keeffe, founder of MeriTalk, said that improving security requires a mind shift from compliance to full risk management. He added: "Agencies need to think about 'big security' alongside big data. Chief data officers need to be on the court. Data is the MVP."