Accelerating Qlik Sense for Big Data Analytics

Qlik Sense is a great tool for bringing data from different sources together. Qlik Sense automatically profiles new data sources, and the associative features in Qlik allow you to easily blend, analyze and visualize data from multiple sources.

What happens when one or more of these data sources becomes very large? While Qlik compresses data so it can hold multiple data sets in memory, if you have billions of rows to analyze from your data warehouse, Hadoop cluster or cloud storage, then Qlik advocates moving the analytical processing to the data.

Qlik’s On Demand App Generation (ODAG)

Qlik and Kognitio query diagram

Introduced in June 2017, Qlik’s On Demand App Generation (ODAG) functionality allows you to build an overview of your big data that is then used as a “Selection App” to focus in on the “detail data” that you want to bring into Qlik Sense to analyze further.

A well designed ODAG solution allows you to seamlessly review the data available, then select the data you want to look at in more detail within Qlik Sense. This “detail data” is then obtained directly from the big data platform holding the data.

The ability to deliver your “on demand” detail data is wholly dependent on the performance of the system that Qlik Sense connects to to obtain the detail data.

When you make your selections in Qlik, do you want to have to “hang around” while the underlying platform generates your ODAG detail dataset defined by your selection? No, we wouldn’t either. Therefore in order to maintain Qlik’s “train of thought” approach to data analysis, the big data system must be highly performant. That’s why we recommend using Kognitio to accelerate your big data analysis.

Enterprise performance over big data

If an enterprise is looking to introduce Qlik Sense’s great functionality to lots of end users then connecting it to a slow, less performant platform for big data might mean that the “On Demand” part of the App Generation just isn’t possible. Your supporting platform needs to give a lightning response for all users over big data.

Kognitio, like Qlik, is in-memory. Kognitio handles big data and processing seamlessly by distributing workload across multiple servers. To a BI application like Qlik, Kognitio behaves like any other data source with the exception that results are produced ultra-fast – even across billions of rows.

Kognitio’s in-memory MPP processing capabilities are specifically designed to support ad-hoc queries such as those that will be produced by Qlik’s On Demand App Generation. You should be given the freedom to make the appropriate selections to get to the detail that is useful. Experimenting with new ideas shouldn’t be something you avoid – just change your selections and generate the new detail data – simple.

Going further with your Qlik detail apps

In its simplest form a Qlik ODAG detail App works by submitting SQL queries to the big data platform after the end-user makes their selection. With Kognitio as the supporting platform, the Qlik ODAG functionality can be used for much more. After your data scientists have created complex analytics in R, Python etc., they can use the Kognitio non-SQL interface to enable your Qlik ODAG detail App to execute these same complex analytics over the data based on your own ODAG selection. This truly on-demand analytics frees up your expensive data science resource to move onto the next project rather than getting stuck re-running the same analysis over and over again.

Get faster answers to bigger questions

Kognitio is a mature in-memory SQL engine that truly unlocks the insights of big data. It was specifically built for massively parallel analytical query processing to provide ultra-high-speed analytics, enabling data analysts and business users to run thousands of complex queries concurrently, bringing a competitive edge to your big data operations.

  • Flexible setup with many different deployment options: on Hadoop, on MapR, on standalone servers or in the cloud. Develop analytics anywhere and move easily.
  • Ability to run simultaneous mixed-workload use cases at the same time.
  • Sophisticated query planning allows queries on large data sets to be distributed to use all available CPU resource to process the query with maximum performance.
  • Parallelizes queries written in any Linux-compatible programming language, taking advantage of Kognitio’s massively parallel in-memory processing capabilities to deliver high performance advanced analytics.
  • Mature SQL implementation capable of running all TPC-DS queries as well as providing functions to allow non ANSI standard SQL from other vendors to run unchanged.
  • A wide variety of standard connectors which make connecting to any data source simple and transparent whether the source is in a Hadoop cluster, in cloud storage or an EDW. In Kognitio they all appear as tables to the end-user.

Get started with Kognitio today.