Case studies

Global credit card case study

How does the world’s largest payment card issuer enable 10,000 Tableau users to analyze data held in a nine petabyte Hadoop cluster in near real-time? With Kognitio.

Billions of transactions

With billions of transactions flowing through their network, the world’s largest provider of payment cards is a leader in capturing, analyzing and gaining insight from that data. As one of the world’s few end-to-end providers, this data covers each step of the payment process from merchant through to bill payment.

As the largest provider of company charge cards, the business wanted to share that insight with their corporate clients around the world. However, having 100,000 such clients brings with it unique challenges.

Highly concurrent complex queries

Understanding the spending patterns of hundreds, maybe thousands, of employees is a full-time job at many large businesses. Making it easy to get that insight is one of the reasons why the world’s largest corporate card provider remains a favorite with such clients.

The challenge of providing that insight was on a scale that pretty much no tool could handle. Here’s why: the provider knew that they’d have three to four hundred clients using the system at any one time. The dataset covered twelve billion transactions held in an enormous mixed-use Hadoop cluster. Their preferred visualization tool, Tableau, would generate up to 100 complex SQL queries per page rendered.

Well known Hadoop query tools, whether SQL-based or Spark, wouldn’t be able to even approach real-time rendering. And that’s before considering the incomplete SQL coverage that lets down most Hadoop ecosystem SQL query engines.

One alternative was to shard their data into individual extracts for each client. Maintaining 100,000 near real-time extracts would be operationally impossible and add expense to onboarding new customers.

The option they chose was to use a query engine for Hadoop that could handle hundreds of concurrent complex SQL queries over the entire data set and return the results in near real-time. That was Kognitio.

Kognitio on Hadoop

The team at Kognitio provided an ultra-high-performance, high-concurrency, in-memory SQL query engine that sits between the Hadoop cluster and the Tableau front-end.

Kognitio integrates seamlessly with the Hadoop cluster, pulling the 12 billion transactions from HDFS (Hadoop file system) directly into memory. For each view that it needs to render, Tableau sends its SQL queries directly to Kognitio, which returns the results in near real-time.

Delivering what other solutions cannot

Using Kognitio as an acceleration layer between Tableau and Hadoop has allowed the company to deliver a solution that provides their customers with market leading analytics capabilities.

The extreme performance of the Kognitio layer gives them the confidence to offer this solution to all of their corporate clients, sure in the knowledge that the tools will remain responsive at the highest levels of usage. This solution maintains the company’s reputation as the most innovative of credit card providers. Many other “SQL on Hadoop” solutions were tried and proved incapable of providing the functionality, reliability and performance necessary to deliver an enterprise class solution.