Starting and stopping a Kognitio on Hadoop system¶
To quickly stop and start a Kodoop database at the server level, the following commands can be used. These will control a database server running within a YARN application cluster, but will not affect the cluster itself.
kodoop server <id> start. This will also restart the server if it is already running
kodoop server <id> stop
To stop and start a Kodoop cluster, the following commands can be used. These control the YARN application cluster containing a server.
kodoop cluster <id> stop. Stops the cluster’s yarn application. This will shut down everything except the edge nodes. Memory images will be lost but internal data will persist in HDFS.
kodoop cluster <id> start. Starts up the cluster. Starts a YARN application and containers.
kodoop cluster <id> restart. Stop and then start again.
Note that Kognitio will not check for active user sessions when you stop the system, so care should be taken not to interrupt important queries. Failure to do this may temporarily affect the performance or availability of Kognitio when it is restarted, as any incomplete operations may have to be rolled back or resumed.
The rest of this document goes into more detail on other startup options.
When a Kodoop system is started, there are 3 ways in which table and view images may be treated, depending on the situation:
recover old images: The system may quickly restore images that were in place before the startup, using PMA’s (see Fast Restarts below)
rebuild images: The system may rebuild all images from disk. This is much slower than recovering images, but is necessary in some situations where recovering is not possible
rebuild system table images: The system may leave user tables and views un-imaged and only rebuild system table images from disk
Recovering images will not be possible if:
one or more of the server containers has been recreated, for example after running
kodoop cluster system_id stopand
kodoop cluster system_id start
there have been changes to the size and/or number of ram stores (e.g. adding new containers)
Many of the subsequent startup commands in this document require the use of a special shell which is configured to run management layer commands for a specified cluster. This shell can be invoked on any edge node with the command
kodoop mgr <system_id> shell. It is designed for compatibility with legacy standalone commands such as
wxserver start which do not specify a system ID.
Recovering Old Images
In some situations it is possible to speed up system restarts, by reusing the memory the server was using prior to the restart, rather than having to load all objects into memory from disk. This is done via PMA areas.
PMAs (Persistent Memory Areas) contain table and view image data. They occupy a percentage of the RAM available for in-memory processing in a Kodoop cluster (by default 85%). The remaining RAM is used for transient data such as temporary tables created by the query optimiser, sort buffers, and so on.
To recover images (otherwise known as a ‘fast restart’), you can either run
kodoop server <system_id> start from an edge node, or run
wxserver start within a management shell. This command will attempt a fast restart, and fall back to rebuilding images if this is not possible. By default, this is equivalent to the command
wxserver start recover or image which can also be run within a management shell.
To attempt a fast restart, and error if that fails, run
wxserver start recover within a management shell.
To attempt a fast restart, and rebuild only system table images if that fails, use
wxserver start recover or sysimage within a management shell.
The percentage of RAM used for PMAs is set with the rs_core_pma_size parameter in the [boot options] section of the configuration file.
If image recovery is not desirable or possible, you can start a system without image recovery by running the following command within a management shell:
``wxserver start [sysimage] without recovery``
sysimage will rebuild only system table images. If
sysimage is not specified, all images will be rebuilt.
Rebuilding System Table Images
To start a system by rebuilding only system table images, run
wxserver start sysimage within a management shell.
Configuration File Options
Parameters can be set in the
[boot options] section of the config file, which control behavior when
wxserver start or
wxserver start sysimage are run:
auto_recover_on_image=[yes|no]will cause the
wxserver startcommand to attempt recovery. This defaults to ‘yes’.
auto_recover_on_sysimage=[yes|no]will case the
wxserver start sysimagecommand to attempt recovery. This defaults to ‘no’.
require_auto_recover option=[yes|no]causes all restarts to attempt recovery unless they explicitly have “without recovery” appended. This will also default to ‘no’.