Now, I’ve just read the website from a company called Splunk. They were on BBC Radio 4’s Today programme recently talking about big data – a subject close to my heart. So, I thought I would look at this interesting analytical company and find out more. Well blow me down sideways if I only went and found this on their website:

Traditional technologies, built on relational or multi-dimensional databases, cannot handle the complexity or scale of today’s massive volumes of machine data. Nor do they allow the flexibility to ask any question or get questions answered in real time–which now is an expectation of users.
Eureka, I thought, someone who understands. They have clearly been burnt in the past. I looked more at their website and their Operational Intelligence solutions. Their philosophy completely aligns with my own and what’s more they have made a highly successful business. So, thirsty for more, I hunted around trying to find more about the root cause of the trouble Splunk addresses: the inability of relational or multi-dimensional databases to support users’ very detailed, very diverse and very eclectic questions. Here at Kognitio I call it “Creative Curiosity” as the hunt goes on for the cause of trouble or the germ of opportunity. And it was at this point I got lost. Not their fault, I just found it hard to understand their file indexing approach in relation to using a traditional relational database. It clearly works, but I wanted to go and see them and say, I’ve got a relational and multi-dimensional database that can deal with complexity and scale and it is by no means traditional. I can get data to scale out really rapidly and I can allow customers to dive around the data to their heart’s content, why don’t we work together, I thought.
Never mind, at least I know there are others out there who understand this point. There is no longer a need to struggle with the data. My colleagues and I have been pressing hard with the message that business people can have all the detail they want on database platforms like ours at Kognitio. Kognitio do this by putting the data in memory and then massively parallel processing it with lots and lots and lots of processing cores. Nowadays that is economic to do because server prices and RAM charges have fallen so far.
But, it’s not only the technology. There’s this whole cloud thing too. I am blown away by the amazing things that people are doing on the cloud and thanks has to go to Gartner for categorizing this all so neatly in to those people who offer “Software as a Service”, “Infrastructure as a Service” and “Platform as a Service” (amongst other “as a service”). The cloud sets you free. Big procurement exercises to ensure database capacity are a thing of the past if you are willing to put the data in the cloud. You simply buy the capacity and performance you want. It’s all there. Like all revolutions, it spreads and changes quickly, but I think Splunk are right “Traditional technologies….cannot handle the complexity and scale of today’s massive volumes”, the questions is how long will it take the new options to be accepted.