General discussion on using the Kognitio Analytical Platform.

Using compressed RAM images

by deborahmartin » Wed Jul 26, 2017 12:02 pm

- Are there any known issues with using compressed RAM images?

We don't know of any issues. If you do have an issue please report it to Kognitio using the methods available to you.
These are the Kognitio Support Portal if you have named account portal access, Kognitio forums which are free to
use or send an email to the helpdesk at Kognitio.

- How is a compressed RAM image used?

Simply use the 'compressed' keyword at the end of an image creation statement. Compressed images can be used in the same way as
regular images and can be updated, inserted into, etc the same way as regular images.

- What compression ratio is achievable and what factors influence the level of compression?

The compression is row based and uses the LZ4 encoding so it can get anywhere between 1.5:1 and 5:1 depending on the data.
It tends to work best on wide tables with a lot of true/false attribute columns.

- What is the performance overhead of using compressed RAM images?

It depends on the compression ratio. The better the ratio the lower the performance overhead. Creating and scanning
compressed images can be significantly slower for tables with a low compression ratio. At a 2:1 ratio (50% saving) the
scanning overhead will be significant. This is less of a problem for complex queries than for simple ones because
complex queries spent a much lower percentage of their time scanning data. The scanning overhead can be mitigated in
many cases through the use of appropriate partitioning. Partition elimination is done before decompression so only partitions
hit by a query get decompressed.

- Is the whole image uncompressed prior to use?

No. We uncompress each block at a time. Some parts of the query engine will hold multiple uncompressed blocks at once
but these count as temporary query buffers and the total memory used by these scales with workload and available memory.

- Is each row uncompressed separately to perform a join?

No. The join algorithm is efficient so it only ever performs linear scans and uncompresses blocks. We never uncompress
single rows. Temporary uncompressed blocks described above are indexed via a hash table to perform a join. Joins between
appropriately partitioned tables are often able to eliminate partitions prior to the join, removing the need to decompress
parts of the table.

- Are columns compressed individually such that columns required for joining can be uncompressed as required?

No. Compression is row based. Columnar compression is being worked on.

- Are join columns from the other object compressed to perform an equi-join?”

- No. Joins work by decompressing columns
Reply with quote Top

Who is online

Users browsing this forum: No registered users and 1 guest