What’s new in Kognitio 8.2.4?

The current version of Kognitio is 8.2.4.

In this article we cover:

New features

  • Backup and restore to AWS S3

    Two new client tools are now available for direct backup into AWS S3. These work in a similar way to existing hdfs backup tools released in 8.2.3.

    wxs3backup performs a backup directly into S3. Temporary local storage is used for metadata only. Once the backup is complete this is tarred up and transferred to S3 as well.

    wxs3restore unpacks a metadata backup tarball, created by the wxs3backup process and a restore is then carried out transferring any data directly from S3 files.

    For more details and examples see the Backup to HDFS and S3 reference sheet

  • Improved set of MacOS client tools and ODBC drivers

    Two new drivers for connecting to Kognitio via ODBC are available with the release of 8.2.4. These use either the unixODBC or iODBC driver managers. See the MacOS section of the Kognitio ODBC driver installation page for more details.

    We have also improved the operability of Kognitio Console for MacOS. A new version is available to download from Kognitio Client Tools Download Page.

  • Enhanced endpoint verification capabilities for SSL client connections

    We have overhauled the SSL verfication process in Kognitio for JDBC connections to keep in line with the behavior of newer Java Runtime Environments. This means when you uprade to Kognitio 8.2.4 onwards you must make sure that you also upgrade the JDBC driver available from the Kognitio Client Tools Download Page as well. The legacy JDBC driver (version 8.2.3 or earlier) will no longer work.

    See security changes in behavior for more details.

  • JDBC connector

    This new connector provides access to any third-party data source that supports a JDBC connection such as AWS Aurora, AWS Redshift, Spark or Presto. Access data you have stored elsewhere and bring it into Kognitio RAM for faster processing allowing you to easily offload analytical workloads.

    The data source must be reachable from all Kognitio nodes and the JDBC driver you specify in the connector must be located somewhere Kognitio knows to look for it. For more details see JDBC connector quick reference sheet.

Changes in behavior

This section outlines the changes in behavior in Kognitio version 8.2.4 when compared with previous Kognitio 8.2.3 release

Performance Improvements

  • Readahead caching for flat files in S3 and HDFS connectors - to improve read performance the S3 and HDFS connectors now fetch blocks of data size specified by the bl_cache_size parameter (default 1MB). The parameter is expressed in bytes and can be set in the range 0 to 10 MB.

  • Prefix optimisation for wildcard file specification in S3 - when a uri_path attribute contains a wildcard better filtering of S3 objects has been implemented. This improves peformance when there are a large number of files in a bucket.

  • Binary encoding between ORC/Parquet connector and Kognitio - improved handling of binary encoding means some ORC/Parquet files are now read significantly faster by Kognitio. Improvements mean load times are below 50% of previous timings in some cases.

  • Performance Improvements on some aggregate functions and TOP N queries

External Data Sources

  • Writable external table support for CSV files in S3 - external tables using flat file (CSV) format created in S3 can now be made writable.

  • AWS Glue Connector can now create new tables in Glue - a new target attribute create_table has been added to the Glue Connector instructing it to create a table in AWS Glue based on the data location and column definitions provided. To ensure no existing objects are overwritten if an object already exists at the data location provided an error is returned. For more details see the AWS Glue Connector quick reference sheet.

  • S3 connector now support requester-pays buckets - a new requesterpays target attribute has been added to the S3 connector, (default value 0). To list or access requester-pays buckets in AWS users must indicate they are aware they may be charged for the request. If you want to use a requester-pays bucket from Kognitio you must set requesterpays 1 in the target string to indicate to S3 that you are happy to be charged for accessing this data source. See S3 connector quick reference sheet for more details.

  • Parquet and ORC coneectors now truncate strings that are too long - the target attribute fmt_truncate_strings that silently truncates any strings too long to fit in CHAR or VARCHAR columns can now be applied to Parquet and ORC data sources. see target string attributes quick reference sheet for more details.

  • Support for TIME data type in Glue Connector

  • Parquet Connector correct support for sub-second times

Security

  • Client connections now use SSL by default - connections no longer transparently fall back to plain unencrypted text. Users must explicitly disable SSL in order to use plain text connections. This has implications for JDBC endpoint verification.

  • Support of endpoint verification for SSL connections and JDBC - From Kognitio 8.2.4 onwards SSL is used by default. For JDBC connections this means you must add the server’s SSL certificate to your trust store or choose to disable SSL. For more details see JDBC and SSL certificates quick reference sheet Note in earlier versions of Kognitio this was not required; a legacy anonymous key-exchange method was used. This is no longer supported by newer Java environments.

  • Improved security of host process for Java based connectors - changes to all Java based connectors have been carried out to keep inline with the support of endpoint verification and default use of SSL.

  • Kognitio to Kognitio connectors using ODBC now use SSL - when one Kognitio system is defined as an external data source for a another any ODBC connections between the systems now automatically use SSL. This applies to the Crossload, Unloader and standard ODBC connectors.