What’s new in Kognitio 8.2.4?¶
The current version of Kognitio is 8.2.4.
In this article we cover:
Backup and restore to AWS S3
Two new client tools are now available for direct backup into AWS S3. These work in a similar way to existing hdfs backup tools released in 8.2.3.
wxs3backupperforms a backup directly into S3. Temporary local storage is used for metadata only. Once the backup is complete this is tarred up and transferred to S3 as well.
wxs3restoreunpacks a metadata backup tarball, created by the
wxs3backupprocess and a restore is then carried out transferring any data directly from S3 files.
For more details and examples see the
Backup to HDFS and S3 reference sheet
Improved set of MacOS client tools and ODBC drivers
Two new drivers for connecting to Kognitio via ODBC are available with the release of 8.2.4. These use either the unixODBC or iODBC driver managers. See the MacOS section of the Kognitio ODBC driver installation page for more details.
We have also improved the operability of Kognitio Console for MacOS. A new version is available to download from Kognitio Client Tools Download Page.
Enhanced endpoint verification capabilities for SSL client connections
We have overhauled the SSL verfication process in Kognitio for JDBC connections to keep in line with the behavior of newer Java Runtime Environments. This means when you uprade to Kognitio 8.2.4 onwards you must make sure that you also upgrade the JDBC driver available from the Kognitio Client Tools Download Page as well. The legacy JDBC driver (version 8.2.3 or earlier) will no longer work.
See security changes in behavior for more details.
This new connector provides access to any third-party data source that supports a JDBC connection such as AWS Aurora, AWS Redshift, Spark or Presto. Access data you have stored elsewhere and bring it into Kognitio RAM for faster processing allowing you to easily offload analytical workloads.
The data source must be reachable from all Kognitio nodes and the JDBC driver you specify in the connector must be located somewhere Kognitio knows to look for it. For more details see
JDBC connector quick reference sheet.
Changes in behavior¶
This section outlines the changes in behavior in Kognitio version 8.2.4 when compared with previous Kognitio 8.2.3 release
Readahead caching for flat files in S3 and HDFS connectors - to improve read performance the S3 and HDFS connectors now fetch blocks of data size specified by the
bl_cache_sizeparameter (default 1MB). The parameter is expressed in bytes and can be set in the range 0 to 10 MB.
Prefix optimisation for wildcard file specification in S3 - when a
uri_pathattribute contains a wildcard better filtering of S3 objects has been implemented. This improves peformance when there are a large number of files in a bucket.
Binary encoding between ORC/Parquet connector and Kognitio - improved handling of binary encoding means some ORC/Parquet files are now read significantly faster by Kognitio. Improvements mean load times are below 50% of previous timings in some cases.
Performance Improvements on some aggregate functions and TOP N queries
External Data Sources¶
Writable external table support for CSV files in S3 - external tables using flat file (CSV) format created in S3 can now be made writable.
AWS Glue Connector can now create new tables in Glue - a new target attribute
create_tablehas been added to the Glue Connector instructing it to create a table in AWS Glue based on the data location and column definitions provided. To ensure no existing objects are overwritten if an object already exists at the data location provided an error is returned. For more details see the
AWS Glue Connector quick reference sheet.
S3 connector now support requester-pays buckets - a new
requesterpaystarget attribute has been added to the S3 connector, (default value 0). To list or access requester-pays buckets in AWS users must indicate they are aware they may be charged for the request. If you want to use a requester-pays bucket from Kognitio you must set
requesterpays 1in the
target stringto indicate to S3 that you are happy to be charged for accessing this data source. See
S3 connector quick reference sheetfor more details.
Parquet and ORC coneectors now truncate strings that are too long - the target attribute
fmt_truncate_stringsthat silently truncates any strings too long to fit in
VARCHARcolumns can now be applied to Parquet and ORC data sources. see
target string attributes quick reference sheetfor more details.
Support for TIME data type in Glue Connector
Parquet Connector correct support for sub-second times
Client connections now use SSL by default - connections no longer transparently fall back to plain unencrypted text. Users must explicitly disable SSL in order to use plain text connections. This has implications for JDBC endpoint verification.
Support of endpoint verification for SSL connections and JDBC - From Kognitio 8.2.4 onwards SSL is used by default. For JDBC connections this means you must add the server’s SSL certificate to your trust store or choose to disable SSL. For more details see
JDBC and SSL certificates quick reference sheetNote in earlier versions of Kognitio this was not required; a legacy anonymous key-exchange method was used. This is no longer supported by newer Java environments.
Improved security of host process for Java based connectors - changes to all Java based connectors have been carried out to keep inline with the support of endpoint verification and default use of SSL.
Kognitio to Kognitio connectors using ODBC now use SSL - when one Kognitio system is defined as an external data source for a another any ODBC connections between the systems now automatically use SSL. This applies to the Crossload, Unloader and standard ODBC connectors.