Recovering disk space

Linear file system

Kognitio implements a simple linear file system. The method of updating data is to write a new version, and indicate that the previous record has been superseded. This concept of not updating in place provides many benefits including:

  • Fast commit / rollback processing

  • Consistent read-only database scans—no interference with concurrent updates

This means that data is not physically deleted from the disk when a database record is deleted, or when a table is dropped or truncated. Two methods are available to recover disk space from the Kognitio file system: Reclaiming and Repacking. During both operations user data that is still valid is concatenated into one contiguous chunk, and all deleted and rolled back records are removed. Repack is generally recommended rather than reclaim, since repack is more granular, and does not require a global system lock.

Note that disk space used by system tables is a special case. See here for more information on how to recover disk space used by system tables.

Reclaim

The RECLAIM command lets the system administrator reclaim disk space currently occupied by records that have been marked as deleted or rolled back in transactions.

The disadvantages of RECLAIM compared to REPACK are that RECLAIM requires a system lock, so no other sessions can be running during a reclaim. Also, at the end of a reclaim, all images must be recreated.

The advantage of reclaim over repack is that it is significantly faster than REPACK would be if repacking tables with images.

Reclaiming space may take several hours, so Kognitio recommend that you timetable reclaims on a regular basis and run them overnight or at weekends. Individual disk slabs can be specified if there is only a small daily reclaim window available meaning a different set of slabs have to be reclaimed each night; alternatively you may wish to target an individual slab because you have deleted a significant number of records from it, for example, you may have deleted old logging records from the logging slab.

By default, RECLAIM reads in blocks of data, and only starts recovering space for a slab when it finds a block whose percentage of reclaimable disk space is greater than the ds_gsr_percent parameter (default value 80). From that point onwards, all possible space is recovered. The reason for the parameter is to prevent a reclaim recovering one record early in the slab, and then having to shuffle all the remaining data in the slab down by one record, which would be very expensive for little benefit.

By specifying the --expect-incremental flag to a backup, RECLAIM will not remove any data required by a subsequent incremental backup; if the flag is not set running a RECLAIM will prevent further incremental backups and require the next backup to be a full backup.

Compressed Data Maps are removed when a RECLAIM is run.

Running a Reclaim

Before running a RECLAIM, The SYS user must obtain a global session by locking the system:

LOCK SYSTEM;

The RECLAIM command tries to get a global lock implicitly if its session does not already have one. The global lock will be held until the end of the reclaim session.

The syntax for reclaim is:

RECLAIM [FOR PARTITION {slab_id} ...] TO {NOW | BEGIN}

Note that if the global lock cannot be obtained, the RECLAIM will not be run.

Typically reclaim will be scripted to start a loop to abort all other sessions and get a global lock, with the loop only exiting when the global lock is successfully obtained. Then the script will move on to run the reclaim command.

How much will be Reclaimed?

The virtual table ipe_ftable has information on active and dropped tables, including the amount of space occupied by deleted and truncated rows, as well as the space used by active rows.

  • DEL_ROWS indicates how many deleted rows exist.

  • DEL_WORDS indicates how many deleted words those rows occupy.

  • TRUNC_WORDS indicates how many words are occupied by truncated rows from the table.

  • If DROP_TNO is less than 2147483647, this indicates the table has been dropped.

For example, the following query gives an estimate of recoverable data

SELECT SUM(t) MB_reclaimable
FROM (SELECT (CASE WHEN drop_tno=2147483647 THEN del_words+trunc_words
                   ELSE words+del_words+trunc_words
              END) /(1032*1032) as t
      FROM ipe_ftable) df;

Repacking

Before reading this section, it’s important to understand how slabs work in Kognitio, and how tables can be assigned to particular slabs.

REPACK is an operation which removes deleted data from a system, similarly to RECLAIM.

The disadvantages of RECLAIM compared to REPACK are that RECLAIM requires a system lock, so no other sessions can be running during a reclaim. Whereas REPACK is designed to be run in the background, and to not interrupt normal usage of the system. Also, at the end of a reclaim, all images must be recreated, which is not the case with a repack. The advantage of reclaim over repack is that it is significantly faster than repack would be if repacking tables with images.

How Repack Works

When you run repack, you will specify a list of slabs to be repacked. The syntax used to run repack manually is:

REPACK SLABS slab_list [ROTATE]

where slab_list is a comma separated list of slab numbers.

When the above command is run, Kognitio will repack each slab in the supplied list, one at a time. For each slab, REPACK will make a list of every table which has non-deleted rows on that slab. It will then process the tables one at a time. For each table, REPACK will check whether any other session has any kind of lock on the table. If so, REPACK will skip the table and move on to the next table in it’s list. If there are no locks, REPACK will attempt to move all of the table’s non-deleted rows onto other slabs which the table is assigned to. When this is complete, repack will move onto the next table in the list. If another session requests a lock on a table while it’s rows are being moved, REPACK will back off and move to the next table.

When all tables on the slab have been considered, REPACK will check to see if any non-deleted rows remain on the slab. If there are no non-deleted rows, repack will zero the slab completely, wiping all data, and then move on to the next slab. However, if REPACK finds non-deleted rows on the slab, it will move on to the next slab without doing any zeroing.

This means that when REPACK is run on a system with concurrent active sessions, there is no guarantee that the slabs you specify will be successfully zeroed. Additionally, REPACK will not return an error if any slabs were not zeroed, as it runs in the background.

There are two main reasons that REPACK may fail to zero a given slab:

  1. another session had a lock on a table which has live rows on the slab, when REAPCK tried to process it. OR

  2. there was a table on the slab whose live rows could not all be moved to other slabs in the table’s assigned slab list. This could be because the table did not have other assigned slabs, or it could be because there was insufficient free space on the other assigned slabs.

If there are tables in the system which are only assigned to one slab, it is still possible to repack that slab if ROTATE is appended to the repack command. The ROTATE option tells REAPCK to move live data across disk resources when preparing to zero a slab, instead of moving it to other assigned slabs. In this case, REPACK will zero the section of the slab on one disk resource, then move onto the section of the slab on the next disk resource, and so on.

Note that while REPAXK is running, system performance may be degraded, particularly for disk-heavy queries. Also note that slab 1, which is reserved for system tables, can only be repacked if it is the only slab in the specified slab list. The same applies to slab 2 (the logging slab). Meaning that this is possible:

REPACK SLABS 1;
REPACK SLABS 2;

But neither of these queries are possible:

REPACK SLABS 1,2;
REPACK SLABS 1,2,3,4;

Automated Repacking

Kognitio also has an automated background repacker which can be turned on if needed. The background repacker looks for disk usage getting above a certain percentage on any disk resource, and when this threshold is reached, the repacker processes the disks one slab at a time; all the tables from the slab are repacked and the slab is left empty and ready for reuse. If a table could not be repacked as it can only exist in the current slab, the repacker moves onto the next slab.

The background repacker is controlled through the system parameter “repack_level” which is used to specify the percentage of disk resource that should be used before repacking takes place. The default value is 101, which means that background repacking is off by default.

Some examples of where the background repacker can be useful are:

  • If a set of slabs are assigned to an ETL staging area and nothing else uses the slabs, then once the data has been loaded, manipulated and inserted into the target tables the staging data can be dropped/truncated and Kognitio will then automatically free up the slabs ready for the next load.

  • A large fact table is to be recreated on a monthly basis; the initial table is exclusively assigned to a set of slabs, the following month the table is exclusively assigned to a new set of slabs and recreated, the previous table can then be dropped/truncated and Kognitio will automatically free up the slabs ready for the next iteration.

Note that:

  • Whenever a REPACK runs, the details of any associated RAM images are updated automatically.

  • Compressed Data Maps are unaffected by repacking.

  • Running an explicit RECLAIM will remove any Compressed Data Maps.

  • Data can still be accessed while a repack is going on. The repacker can be interrupted if data is inserted into a table that is being repacked; the interruption does not cause the existing work to be rolled back and repacking will resume automatically.

  • The activity of the background repacker is visible in the IPE_CURSESSIONS and IPE_CURTRANS system tables; this allows the user to see which tables are being repacked, and also abort the operations.