Preparing MapR for a Kognitio install

Attention

 

To deploy Kognitio on MapR systems we now recommend using Containerized Kognitio .

Therefore the “Kognitio on MapR” product is no longer available to new users.

Existing users can still receive updates via support channels. Please contact us via the community forum or our support portal if you want to migrate to a containerized deployment.

 

This section shows how to install Kognitio 8.2.0 on MapR.

Prerequisites

Before you install Kognitio on MapR:

  • Check that you have MapR running. Kognitio can run on MapR version 5.2.2, with or without security, and MapR version 6.0.0. In our examples the MapR cluster is called kogmapr.

  • Make sure that you have MapR-FS storage set up with a “standard” MapR NFS mount for your MapR cluster. The NFS must be mounted on all nodes where you want to run Kognitio. In our example this is located at /mapr/kogmapr.

  • Install the prerequisite libraries required for Kognitio.

  • If you intend to use more than half the available RAM for Kognitio, adjust the limits of /dev/shm filesystem to allow Kognitio to use memory effectively.

  • Configure MapR to allocate memory to Kognitio.

Preparation

Perform step 3 as the mapr user. Perform all the other steps as root.

1. Create a Linux group and user for Kognitio.

On every node, create a group and a user to install Kognitio on MapR. In these examples we use “koggrp” and “kogadmin”.

When the installation is finished, you’ll use kogadmin for system administration tasks.

Add kogadmin to the mapr group. MapR is a prerequisite, so you should already have a mapr group.

groupadd -g <kog group num> koggrp
useradd -c "Kognitio Service User" -u <kog user num> -d /home/kogadmin -g koggrp -G mapr -m kogadmin

Choose values for <kog group num> and <kog user num> that are not already used. Use the same values on every node.

2. Set user limits.

For Kognitio to run effectively, you need to increase the nproc default limit for kogadmin. Add this line to /etc/security/limits.conf:

kogadmin soft nproc 100000

3. Create a service ticket for the kogadmin user.

If you are running an unsecured MapR version 5.2.2 cluster, you can skip this step.

Log on as the mapr user, run maprlogin, then run:

maprlogin generateticket -type service -user kogadmin -out <kognitio-ticket-file>

Copy the output file <kognitio-ticket-file> to /tmp/maprticket_<kog user num> on every node.

On every node, make the kogadmin user the owner of this file and restrict access to the file. In our example:

chown kogadmin:koggrp /tmp/maprticket_<kog user num>
chmod 600 /tmp/maprticket_<kog user num>

4. Create a shared volume in MapR-FS to hold the Kognitio software download and node registry files.

If you are installing on secure clusters, run maprlogin before you begin this step.

On one of the nodes only, create a MapR-FS volume for Kognitio. In this example, we call this Kognitio volume: kogvol01. Give kogadmin full control of the MapR-FS volume.

maprcli volume create -name kogvol01 -path /kogvol01
maprcli acl edit -type volume -name kogvol01 -user kogadmin:fc

5. Create some directories on the MapR-FS volume (accessible from every node) and make kogadmin the owner.

Create a directory to hold the downloads:

mkdir -p /mapr/kogmapr/kogvol01/downloads

Create a directory where Kognitio can write node registration information:

mkdir -p /mapr/kogmapr/kogvol01/nodereg

Create a directory to hold Kognitio data:

mkdir -p /mapr/kogmapr/kogvol01/dfs

Make kogadmin the owner of these directories:

chown -R kogadmin:koggrp /mapr/kogmapr/kogvol01/

6. Create a MapR-FS volume on every node in the system and enable access.

You can do this from a single node via the maprcli utility. If you are on a secure cluster, make sure you run maprlogin before you use maprcli.

To find the nodes, run:

maprcli node list -columns name

This gives you the names and ip addresses of the nodes in your system. We have four AWS nodes in our MapR cluster:

hostname                                  ip
ip-10-4-1-11.eu-west-1.compute.internal   10.4.1.11
ip-10-4-1-98.eu-west-1.compute.internal   10.4.1.98
ip-10-4-1-126.eu-west-1.compute.internal  10.4.1.126
ip-10-4-1-192.eu-west-1.compute.internal  10.4.1.192

Run the following commands for all the nodes. You can do this from a single node and change -localvolumehost to each CLDB node in turn:

maprcli volume create -name <kognitio local vol name> -path /kogvol01/dfs/<kognitio local vol name> -createparent true -localvolumehost <local vol name> -replication 1
maprcli acl edit -type volume -name <kognitio local vol name> -user <kog user>:fc

For the 10.4.1.11 node, for example:

maprcli volume create -name kognitio-volume-ip-10.4.1.11 -path /kogvol01/dfs/kognitio-volume-ip-10.4.1.11 -createparent true -localvolumehost 10.4.1.11 -replication 1
maprcli acl edit -type volume -name kognitio-volume-ip-10.4.1.11 -user kogadmin:fc

It is good practice to choose volume names that match the nodes where they are located. For the other three nodes in our system:

maprcli volume create -name kognitio-volume-ip-10.4.1.98 -path /kogvol01/dfs/kognitio-volume-ip-10.4.1.98 -createparent true -localvolumehost 10.4.1.98 -replication 1
maprcli acl edit -type volume -name kognitio-volume-ip-10.4.1.98 -user kogadmin:fc
maprcli volume create -name kognitio-volume-ip-10.4.1.126 -path /kogvol01/dfs/kognitio-volume-ip-10.4.1.126 -createparent true -localvolumehost 10.4.1.126 -replication 1
maprcli acl edit -type volume -name kognitio-volume-ip-10.4.1.126 -user kogadmin:fc
maprcli volume create -name kognitio-volume-ip-10.4.1.192 -path /kogvol01/dfs/kognitio-volume-ip-10.4.1.192 -createparent true -localvolumehost 10.4.1.192 -replication 1
maprcli acl edit -type volume -name kognitio-volume-ip-10.4.1.192 -user kogadmin:fc

The next step is to install and configure Kognitio on MapR.