This blog has been updated to reflect the reduced prerequisite requirements for Kognitio on HadoopRead More
Why ODAG? Is Qlik Sense’s big data implementation right for a data source like Kognitio?
Why ODAG? You’ve probably heard of Qlik Sense’s ODAG (On Demand App Generation) but do you know why you should use it? Or what value it can bring, not only to BI users but data scientists too? No? Then read on. If you are already familiar with the basics then jump to the fourth section to see why Kognitio and ODAG are a good fit.
What is ODAG?
ODAG is how Qlik Sense presents data at different granularity levels with app templates with queries between. App templates? Yes, they’re a series of empty (no data) apps where only the first one (selection app) has data feeding data into the subsequent ones. Each app can contain multiple sheets to dig at it from multiple angles too.
Take a look at the diagram in Qlik’s documentation for a visual look at the flow.
Why should I use ODAG?
You should if you’re using big data. You won’t need to view all of your data at once and won’t want to either. Why? In my experience, it’s too much to show at once. My dashboard ended up being too cluttered and it was confusing to follow.
A more sensible approach would be to view the data at each level starting from the top with a summarized overview. The user can then make a more informed transition to the next level down. There’s also no need to try and join everything together in one convoluted query to try and view everything at once.
The other concern with the all-at-once approach is speed. How long would it take to return that on big data? Terabytes in size? Probably too long where the user has lost their train of thought/interest.
The flow of apps doesn’t have to be linear either, you can set multiple navigation links for an app to take. This might seem convoluted, but Kognitio can take advantage of this…
What does ODAG need to work well?
In the past Qlik’s approach has been one where the speed of the data source doesn’t matter (data silos) and this works for smaller data sets. But once you step into the big data realm, creating silos is a bit more difficult. There’s so much data that you need a fast data source for it, particularly if users are tweaking their selections for their analysis in real-time.
With ODAG, the querying between apps is run on demand and needs to complete in a reasonable amount of time. Kognitio’s fast in-memory processing using SQL fits this very well allowing a more seamless transition between apps. This also needs to work well concurrently with hundreds of users too because there’s likely to be many BI users.
But SQL isn’t the only thing that can be run during those querying intervals. Kognitio can perform non-SQL based complex analytics with external scripts and these are called upon via SQL. This means that analytics can be delivered on the fly using user selections, something not available by default. And it’s also MPP, you could potentially run hundreds or thousands of models and have results returned just as fast as any complex SQL.
Furthermore, model selection is possible taking advantage of the non-linear navigation links mentioned previously. For example, if I were performing product forecasts, should I apply seasonality or not? Is it different for each product/category? As the designer, I can leave this decision to the user. One button for seasonal forecasts and another for non-seasonal. They could even try both and see which fits better.
Why is this functionality important?
Accessibility – Even people who don’t code can run complex analytics, accessing it with a click of a button.
Reproducible – Updates with new data can be processed easily without having to dedicate valuable data scientist time to rerun it.
This is also possible on data from Hadoop. Kognitio enables BI on Hadoop and with interfaces like ODAG, BI on Hadoop with on demand analytics is now possible.
How can I make my own?
Hopefully, that’s given you a better idea as to why ODAG is used for big data why Kognitio is a good fit for it. Do you have another use for ODAG? Let me know in the comments.
If you’re interested in making your own ODAG app, we’ve documented a step-by-step tutorial to make a generic ODAG application with Kognitio but there’s also a video if you want to see it in action with some retail data.