Using Python in external scripts
Posts: 38
Joined: Mon Jan 06, 2014 10:36 am

Kognitio and Python. Threads and Nodes

by skkirkham » Wed Jan 08, 2014 1:49 pm

Hi all

Here is part 2 of series of topics to introduce Kognitio External Scripting using Python.

Following on from the basics this pdf and accompanying examples look at controlling the script invocations within Kognitio.

The external scripting architecture is very flexible so you can control how many (threads) and where (nodes) you want to run your script processes.

The examples used are for finding an average over all the rows of data. While the simplicity of this example allows us to focus on the Kognitio external script functionality, it is important to remember: if you can code the problem in Kognitio using SQL do it as it will run faster. Why I hear you ask?
1) Kognitio SQL is extremely mature and the software has been designed as MPP from the start. This means the Kognitio SQL optimiser utilises the system's parallelism in the most effective way
2) Kognitio SQL is converted into machine code for more efficient faster code execution.
3) When you write external scripts the code execution must be invoked after which data streams through the invocations. There is an overhead associated with this no matter how efficient your code. This will be particularly noticable for simpler tasks that execute quickly (like averaging)

Here's a challenge - if anyone can create an external script in Kognitio that runs faster than an equivalent SQL query on Kognitio then I would love to here about it.

Note If you haven't done so already you will need to create a python script environment on your Kognitio system

Code: Select all

create script environment PYTHON command '/usr/bin/python';
You can call the environment anything you like but the path must match where python is installed on your Kognitio system. More about setting up environments to follow at a later date.
Reply with quote Top

Who is online

Users browsing this forum: No registered users and 1 guest