Starting and stopping a Kognitio on Hadoop system

To quickly stop and start a Kodoop database at the server level, the following commands can be used. These will control a database server running within a YARN application cluster, but will not affect the cluster itself.

  • kodoop server <id> start. This will also restart the server if it is already running

  • kodoop server <id> stop

To stop and start a Kodoop cluster, the following commands can be used. These control the YARN application cluster containing a server.

  • kodoop cluster <id> stop. Stops the cluster’s yarn application. This will shut down everything except the edge nodes. Memory images will be lost but internal data will persist in HDFS.

  • kodoop cluster <id> start. Starts up the cluster. Starts a YARN application and containers.

  • kodoop cluster <id> restart. Stop and then start again.

Note that Kognitio will not check for active user sessions when you stop the system, so care should be taken not to interrupt important queries. Failure to do this may temporarily affect the performance or availability of Kognitio when it is restarted, as any incomplete operations may have to be rolled back or resumed.

The rest of this document goes into more detail on other startup options.

Imaging modes

overview

When a Kodoop system is started, there are 3 ways in which table and view images may be treated, depending on the situation:

  • recover old images: The system may quickly restore images that were in place before the startup, using PMA’s (see Fast Restarts below)

  • rebuild images: The system may rebuild all images from disk. This is much slower than recovering images, but is necessary in some situations where recovering is not possible

  • rebuild system table images: The system may leave user tables and views un-imaged and only rebuild system table images from disk

Recovering images will not be possible if:

  • one or more of the server containers has been recreated, for example after running kodoop cluster system_id stop and kodoop cluster system_id start

  • there have been changes to the size and/or number of ram stores (e.g. adding new containers)

Management Shell

Many of the subsequent startup commands in this document require the use of a special shell which is configured to run management layer commands for a specified cluster. This shell can be invoked on any edge node with the command kodoop mgr <system_id> shell. It is designed for compatibility with legacy standalone commands such as wxserver start which do not specify a system ID.

Recovering Old Images

In some situations it is possible to speed up system restarts, by reusing the memory the server was using prior to the restart, rather than having to load all objects into memory from disk. This is done via PMA areas.

PMAs (Persistent Memory Areas) contain table and view image data. They occupy a percentage of the RAM available for in-memory processing in a Kodoop cluster (by default 85%). The remaining RAM is used for transient data such as temporary tables created by the query optimiser, sort buffers, and so on.

To recover images (otherwise known as a ‘fast restart’), you can either run kodoop server <system_id> start from an edge node, or run wxserver start within a management shell. This command will attempt a fast restart, and fall back to rebuilding images if this is not possible. By default, this is equivalent to the command wxserver start recover or image which can also be run within a management shell.

To attempt a fast restart, and error if that fails, run wxserver start recover within a management shell.

To attempt a fast restart, and rebuild only system table images if that fails, use wxserver start recover or sysimage within a management shell.

The percentage of RAM used for PMAs is set with the rs_core_pma_size parameter in the [boot options] section of the configuration file.

Rebuilding Images

If image recovery is not desirable or possible, you can start a system without image recovery by running the following command within a management shell:

``wxserver start [sysimage] without recovery``

Specifying sysimage will rebuild only system table images. If sysimage is not specified, all images will be rebuilt.

Rebuilding System Table Images

To start a system by rebuilding only system table images, run wxserver start sysimage within a management shell.

Configuration File Options

Parameters can be set in the [boot options] section of the config file, which control behavior when wxserver start or wxserver start sysimage are run:

  • auto_recover_on_image=[yes|no] will cause the wxserver start command to attempt recovery. This defaults to ‘yes’.

  • auto_recover_on_sysimage=[yes|no] will case the wxserver start sysimage command to attempt recovery. This defaults to ‘no’.

  • require_auto_recover option=[yes|no] causes all restarts to attempt recovery unless they explicitly have “without recovery” appended. This will also default to ‘no’.

The startup directory

Each startup attempt will write log files into a separate directory. These files can be useful in diagnosing problems. See logging for more information.