The world’s largest payment card issuer enables 10,000 Tableau users to analyze data held inRead More
The challenge for Inmar
The UK’s top four supermarket chains generated more than £126 billion in sales during the 2015-2016 financial year. In the US, the supermarket sector as a whole is responsible for more than $600 billion in annual sales with profit margins typically hovering around one to two per cent.
In a world of aggressive competition and low margins, grocery retailers have used coupons and loyalty programs to better understand customer spending habits, increase overall spend and, of course, engender loyalty.
Capturing, storing, and making sense of the data from billions of annual transactions, has become an industry in itself. Over the past thirty-five years, Inmar has been at the heart of that industry, helping both retailers and suppliers to unlock the knowledge from sales data. Today, that involves a web portal where Inmar’s customers can put their data to work.
Andrew Wood, a veteran of the retail data industry, is Director of Advanced Analytics at Inmar. Here’s how he describes the scale of the challenge:
|“As a customer, you go to your local supermarket and purchase a basket of shopping. The retailer records every item that you purchase and groups it together under a unique basket identifier. Alongside that is the store number, the checkout lane you used, the date, your loyalty card number, how much you paid for each item and so on.|
Now think of how many other people also shopped at the store that week. Multiply that by 52 times, then again by the number of stores in the chain. Some retailers work with two or three years of data at a time, so it quickly grows to be billions of rows of data.”
It doesn’t just stop with the basket data either; Inmar stores and cross-references ancillary data that helps put purchasing behavior into context. For example, each item has a product category, manufacturer, brand, price, size and so on.
Why did Inmar choose Kognitio?
High-speed, high-volume analytics are at the heart of Inmar’s business. In particular, the self-service portal gives customers direct access to analyze their data in real time. However, querying their Hadoop cluster using Hive made the “high-speed” part of that hard to accomplish.
Andrew had helped to select Kognitio at a previous company and he knew that it could be the answer for Inmar. During an internal data summit, he introduced Kognitio to his colleagues by running identical queries on Kognitio.
|“On Kognitio, it ran something like 300x faster. It was the exact same query and the same data with the same results.”|
The initial results looked as though Kognitio could make a major difference to Inmar’s query speeds. However, there was something Andrew had to consider before rolling Kognitio out to the business.
|“Our data is stored in the ORC format, whereas at the time Kognitio supported other formats such as Parquet. We didn’t want to have to run a data dump to convert from ORC to Parquet, as that would introduce complexities such as meaning the data available for querying with Kognitio would be out of date.|
The Kognitio team didn’t want that either so they developed an ORC connector that enabled us to plug Kognitio directly into our Hadoop data.”
This connector is now available and fully supported in Kognitio.
Kognitio is integral to Inmar’s Hadoop infrastructure
Today, Kognitio is an integral part of Inmar’s Hadoop infrastructure.
|“Kognitio’s high-speed processing means that we can offer more and more real-time queries through our customer portal. For example, it allows us to offer very detailed targeting that otherwise wouldn’t be possible in a reasonable amount of time.”|
But it’s not just about speed. The ability to run various scripting languages within the Kognitio cluster, for queries that wouldn’t be possible with SQL alone, has enabled Inmar to run more detailed analytics models.
|“Integration with R and Python has given us a much better workflow. Previously we would have needed to export data out of Hadoop and then send it to the scripting language, whereas now it all happens within Kognitio running on Hadoop itself.”|
Inmar continues to use a full-range of Hadoop tools but it’s Kognitio that is helping the company to create new, intelligent products.
|“We’re using Kognitio’s analytics, modeling and clustering capabilities to help build new products, such as our new intelligent targeting tools. With Kognitio, we can segment customers in ways that would have taken vastly more effort before and that lets us build new, responsive products that help retailers and suppliers better understand and serve consumers.”|
While competition will most likely continue to pressure retail margins, Inmar is helping their customers to stay ahead by using Kognitio to help find the value in their data.