Forum

Kognitio Console issues, advice, etc.
Contributor
Offline
Posts: 184
Joined: Wed May 29, 2013 2:10 pm
Location: Bracknell

Console Version 80199s170123

by MikeAtkinson » Wed Jan 25, 2017 9:57 am

This release of Kognitio Console has many new features, enhancements to existing features and bug fixes.

SQL in expression contexts
Up until this release SQL could only occur in Lua statements or assignments. For example in Kog Scripts we can write SQL within the script as a statement or assigning the results table to a variable.

Code: Select all

drop table hr.employee;
t = select * from sys.ipe_user; 
This release of Kognitio Console adds the ability to place SQL in Lua expression contexts. So now within Kog Scripts we can place SQL in most places where a Lua expression may occur.
In the example below the data for the Kognitio users are read from the table sys.ipe_user as a Lua table. The rows element of that table is then passed to the apply function which passes each row in turn to the printRow function. The printRow function prints the NAME element of the row to the console, which is attached to the Kognitio Console Script Variables pane.

Code: Select all

function apply(t, fn)
    for i, row in pairs(t) do fn(row) end
end

function printRow(row) print(row.NAME) end

apply( (select * from sys.ipe_user).rows, printRow) 
This feature currently has the limitation that it is not easy for the parser to determine where the SQL query ends. The code below fails because the ‘,’ character does not end an SQL query so it thinks that the , printRow is part of the query. The failure is at run time when the SQL is parsed on the Kognitio Analytical Platform and the SQL "select * from sys.ipe_user, printRow" errors:

Code: Select all

SQL select * from sys.ipe_user, printRow errors.
function apply(t, fn)
    for i, row in pairs(t.rows) do fn(row) end
end

function printRow(row) print(row.NAME) end

apply(select * from sys.ipe_user, printRow) -- ERRORS: , does not end the SQL query
Instead we use ‘;’ to end the SQL expression:

Code: Select all

apply(select * from sys.ipe_user; printRow) – Note use of ; to end SQL
Of particular use while debugging scripts is the form:

Code: Select all

-- any expression of the form print(<sql query>), e.g.
print(select * from sys.ipe_user) 
which outputs the formatted query results to Kognitio Console “Script Variables” pane.

Lua lfs module
The Lua File System module adds common file system manipulation commands to Lua. The lfs module is adapted from the Kepler project and offers a portable way to access the underlying directory structure and file attributes.

lfs.attributes (filepath [, aname | atable])
Returns a table with the file attributes corresponding to filepath (or nil followed by an error message in case of error). If the second optional argument is given and is a string, then only the value of the named attribute is returned (this use is equivalent to lfs.attributes(filepath)[aname], but the table is not created and only one attribute is retrieved from the O.S.). if a table is passed as the second argument, it is filled with attributes and returned instead of a new table. The attributes are described as follows; attribute mode is a string, all the others are numbers, and the time related attributes use the same time reference of os.time:
  • dev
    on Unix systems, this represents the device that the inode resides on. On Windows systems, represents the drive number of the disk containing the file
  • ino
    on Unix systems, this represents the inode number. On Windows systems this has no meaning
  • mode
    string representing the associated protection mode (the values could be file, directory, link, socket, named pipe, char device, block device or other)
  • nlink
    number of hard links to the file
  • uid
    user-id of owner (Unix only, always 0 on Windows)
  • gid
    group-id of owner (Unix only, always 0 on Windows)
  • rdev
    on Unix systems, represents the device type, for special file inodes. On Windows systems represents the same as dev
  • access
    time of last access
  • modification
    time of last data modification
  • change
    time of last file status change
  • size
    file size, in bytes
  • permissions
    file permissions string
  • blocks
    block allocated for file; (Unix only)
  • blksize
    optimal file system I/O blocksize; (Unix only)
    This function uses stat internally thus if the given filepath is a symbolic link, it is followed (if it points to another link the chain is followed recursively) and the information is about the file it refers to. To obtain information about the link itself, see function lfs.symlinkattributes.
lfs.chdir (path)
Changes the current working directory to the given path.
Returns true in case of success or nil plus an error string.

lfs.currentdir ()
Returns a string with the current working directory or nil plus an error string.
iter, dir_obj = lfs.dir (path)
Lua iterator over the entries of a given directory. Each time the iterator is called with dir_obj it returns a directory entry's name as a string, or nil if there are no more entries. You can also iterate by calling dir_obj:next(), and explicitly close the directory before the iteration finished with dir_obj:close(). Raises an error if path is not a directory.

lfs.link (old, new[, symlink])
Creates a link. The first argument is the object to link to and the second is the name of the link. If the optional third argument is true, the link will by a symbolic link (by default, a hard link is created).

lfs.lock (filehandle, mode[, start[, length]])
Locks a file or a part of it. This function works on open files; the file handle should be specified as the first argument. The string mode could be either r (for a read/shared lock) or w (for a write/exclusive lock). The optional arguments start and length can be used to specify a starting point and its length; both should be numbers.
Returns true if the operation was successful; in case of error, it returns nil plus an error string.

fs.lock_dir(path, [seconds_stale])
Creates a lockfile (called lockfile.lfs) in path if it does not exist and returns the lock. If the lock already exists checks if it's stale, using the second parameter (default for the second parameter is INT_MAX, which in practice means the lock will never be stale. To free the the lock call lock:free().
In case of any errors it returns nil and the error message. In particular, if the lock exists and is not stale it returns the "File exists" message.

lfs.mkdir (dirname)
Creates a new directory. The argument is the name of the new directory.
Returns true if the operation was successful; in case of error, it returns nil plus an error string.

lfs.rmdir (dirname)
Removes an existing directory. The argument is the name of the directory.
Returns true if the operation was successful; in case of error, it returns nil plus an error string.

lfs.setmode (file, mode)
Sets the writing mode for a file. The mode string can be either "binary" or "text". Returns true followed the previous mode string for the file, or nilfollowed by an error string in case of errors. On non-Windows platforms, where the two modes are identical, setting the mode has no effect, and the mode is always returned as binary.

lfs.symlinkattributes (filepath [, aname])
Identical to lfs.attributes except that it obtains information about the link itself (not the file it refers to). On Windows this function does not yet support links, and is identical to lfs.attributes.

lfs.touch (filepath [, atime [, mtime]])
Set access and modification times of a file. This function is a bind to utime function. The first argument is the filename, the second argument (atime) is the access time, and the third argument (mtime) is the modification time. Both times are provided in seconds (which should be generated with Lua standard function os.time). If the modification time is omitted, the access time provided is used; if both times are omitted, the current time is used.
Returns true if the operation was successful; in case of error, it returns nil plus an error string.

lfs.unlock (filehandle[, start[, length]])
Unlocks a file or a part of it. This function works on open files; the file handle should be specified as the first argument. The optional arguments start andlength can be used to specify a starting point and its length; both should be numbers.
Returns true if the operation was successful; in case of error, it returns nil plus an error string.

Improvements to Kog Script execution and debugging

Metatables with a __tostring method have been added to the table returned by an SQL query. This allows formatted printing of those tables.

Code: Select all

t = select * from sys.ipe_user;
print(t) –[[ prints the text below:
|NAME         |ID|DFLT_SCNO|SEC_CLASS|SCH_CLASS|STATUS|PTIME              |
|SYS          |0 |0        |0        |0        |0     |2016-12-21 11:19:57|
|GRP_LOG      |3 |0        |0        |0        |8     |2016-12-21 11:20:15|
|GRP_MONITOR  |1 |0        |0        |0        |8     |2016-12-21 11:20:15|
|GRP_DISKUSAGE|4 |0        |0        |0        |8     |2016-12-21 11:20:15|
|GRP_DBG      |2 |0        |0        |0        |8     |2016-12-21 11:20:15|
]]
This may also be used in the kogscript.exe command line tool.
The SQL error string is added to the table returned from the SQL query, and this is also printed out by the metatable __tostring method

Code: Select all

t = select * from nonexistent;
if t.status ~= 0 then 
    print("Query failed: " .. t.error) --[[ prints the text below:
    Query failed: ERROR: [Kognitio][WX2 Driver][latest] CI3013: Table SYS.NONEXISTENT does not exist, offset 15
    ]]
end

print(t) –[[ prints the text below:
ERROR: [Kognitio][WX2 Driver][latest] CI3013: Table SYS.NONEXISTENT does not exist, offset 15
]]
Breakpoints may now also be placed on Lua statements within the Kog Script, even if the “Trace Kog Script Lines” option is not set. After the breakpoint is hit, single stepping then steps through each Lua line.


Timestamps are now converted to a table with year, month, day, hour, min and sec fields:

Code: Select all

t = select * from sys.ipe_allview_img;

list(t.rows[1].CREATE_TIME) –[[ prints the text below:
lua::number day = 16
lua::number min = 45
lua::number month = 1
lua::number year = 2017
lua::number hour = 10
lua::number sec = 34
]]



New Features
    1. Lua File System (lfs) module for functions: attributes (file info), chdir, currentdir, dir (directory iterator), link (not windows), lock, mkdir, rmdir, symlinkattributes, setmode, touch, unlock, lock_dir.
    2. SQL now allowed in Lua expression contexts.
Enhancements
    1. 1. Use Visual C++ 2015 to compile 64 bit windows Console.
    2. Use SQL statement callbacks from Lua, rather than using line callbacks and parsing the statement to see if it is SQL.
    3. Handle UTF8 (non-LATIN1) characters on Console side for Kog Scripts.
    4. When setting history_mode="sql" don't do callbacks from Lua to Console for Lua lines.
    5. Read up to 1,000,000 rows when “Max rows:” set to -1.
    6. Update to use Qt 5.7.1 for 64 bit Windows and OS X builds.
    7. Improve WebEngineViewer with support for Qt 5.7.x QWebEngine.
    8. Add a Drop and Reconnect action.
    9. Add the error string to the Lua table returned by SQL queries if the query errors.
    10. Allow breakpoints and single stepping through Kog scripts even when Kog Script tracing is turned off.
    11. Use a Lua table with year, month, day, hour, min, sec fields for timestamp values.
    12. Add metatable to format print the table returned by SQL query.
    13. Add metatable to format print rows element of Lua table returned by SQL query.
    14. Add a hidden option to turn on overall macro debugging by the <debugmacro> element in configuration file. You can also enable individual macro debugging by setting debug=“true” in the wx2macro element of the macro itself.
    15. Allow users to participate in Console beta program through configuration dialog.
    16. Add a max_rows Lua global variable (setting this alters the max number of rows returned by a query).
    17. Add a quiet mode to kogscript command line tool (enable with –q)
    18. Add a os.getenvs() Lua function, which returns a table of key, value pairs of environment variables.
Bug Fixes
    1. When running a kogscript the diagnose, picture and explain results are not shown correctly when looking back through query history.
    2. Fix JSON parser so it can be told when input has ended.
    3. Make JSON parser error on arithmetic overflow.
    4. Push correct path in Kognitio extension to Lua searchpath() function.
    5. Fix potential uninitialized variable in JobMonitor::end() function.
    6. Lua does not look for Kog scripts within the current directory on Linux.
    7. Improve cleanup after running Kog Script.
    8. Use QPointer references to Loader and Unloader dialogs to avoid double delete.
    9. Show script variables after Kog script is complete.
    10. Add Lua hook to before we close a Lua context; use it to delete the kog script state object.
    11. Write correct query metadata strings using %lld for 64 bit values.
    12. Reset Lua every time we start from the script start.
    13. Improve kog script variables recursive detection.
    14. Avoid buffer overrun in Lua redirect_fwrite().
    15. Memory used after it is freed in Lu all_required() KogScript enhancement.
    16. Don’t eat single newlines in stdout redirection.
    17. Ensure that ScriptResults knows the script name even when not connected to a database.
    18. Cope with rare “offset <n>” in SQL error, before real error offset.
    19. Use a high priority for SessionQueueProcessor threads.
    20. Turn Setting Started Wizard on by default for new users.
    21. Improve access o Configuration data to get rid of Dr Memory detected potential problems.
    22. Fix various other Dr Memory detected potential problems.
    23. Separate out code running on GUI thread better; call it via the event mechanism from non-GUI threads.
    24. Loading scripts from recent files menu does not work for Linux.
    25. Case 20417: creating a menu with a parent window then adding that menu to another menu may cause a crash in Qt 5.
    26. Session Queue Processor is started unnecessarily when it is created.
    27. Output error message when connection attempt fails.
    28. Query Monitor may monitor the wrong connection.
    29. Fixed some memory leaks.
    30. Improve display of script variables that have changed.
    31. Crash is displaying QueryAnalyzer error.
    32. Turn experimental query builder off for 64 bit MSVC builds as it stops pretty printing working.
    33. Update Console copyright to 2017 and help home page.
    34. Kog script syntax colouring fails for x( (...) ).
    35. Deleting data items may cause holes in their parent's children array, which then cause problems with subsequent inserts (may cause a crash).
    36. Fix some build system warnings (delete of void * pointers).
    37. XML Macros should not be disabled by default for new users of the MSVC 64 bit Console.
    38. Data Audit does not work well with tables/views without images.
    39. Crash in loader due to referencing through deleted dlg qpointer.
Tools
    1. Allow kogscript.exe to run pure Lua scripts (i.e. no KogScript enhancements).
    2. For kogscript.exe, if there is not script and it is run in interactive mode then set sqldebug=3 so that SQL results are output by default.
Documentation
    1. Kog Scripting Guide – add Lua File System module documentation.
    2. Kog Scripting Guide – tidy up XHTML.
    3. Kog Scripting Guide – SQL in Lua expression contexts.
Reply with quote Top
Multiple Poster
Offline
User avatar
Posts: 7
Joined: Fri May 08, 2015 3:59 pm

Re: Console Version 80199s170123

by matfior » Wed Jan 25, 2017 1:50 pm

Hi,
the download link to the latest console is not updated, so I still get the old version.

regards,
Matteo
Reply with quote Top
Contributor
Offline
Posts: 184
Joined: Wed May 29, 2013 2:10 pm
Location: Bracknell

Re: Console Version 80199s170123

by MikeAtkinson » Wed Feb 01, 2017 8:28 am

We have three methods of getting Console:
  1. From the "Where can I find the latest Kognitio Console" topic in this forum http://www.kognitio.com/forums/viewtopic.php?f=5&t=10
  2. Via the Version Check tool within Kognitio Console [1]
  3. From the update website [2]
They may get slightly out of synchronisation, but should all be updated within a day.

[1] To turn on the Version Check tool tick the "Enable check for updates" in the Kognitio Console configuration "Miscellaneous" pane.

[2] The updates website is used by the Version Check tool to download updates, you can do the downloads manually updateInfo.xml gives details about what versions are available, they may be downloaded directly using their URLs e.g. http://updates.kognitiocloud.com.s3.ama ... 170123.exe
Reply with quote Top

Who is online

Users browsing this forum: No registered users and 1 guest

cron