Forum

Information and discussion related to the Kognitio on Hadoop product
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Slider not working

by Ashish79 » Wed Feb 06, 2019 9:45 am

While creating cluster 'test1' i am getting below error . Kindly provide the solution.

Starting slider cluster for test1
Failed to run slider command /home/kognitio/kodoop/slider/bin/slider start kognitio-test1
Reply with quote Top
Contributor
Offline
User avatar
Posts: 386
Joined: Thu May 23, 2013 4:48 pm

Re: Slider not working

by markc » Wed Feb 06, 2019 12:26 pm

As with the the topic you posted, can you do "kodoop testenv" to ensure your environment is setup correctly.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Re: Slider not working

by Ashish79 » Thu Feb 07, 2019 6:18 am

markc wrote:
Wed Feb 06, 2019 12:26 pm
As with the the topic you posted, can you do "kodoop testenv" to ensure your environment is setup correctly.
access to HDFS is ok now and i am getting below message to make sure it can access HDFS now.

KODOOP_HOME: /home/kognitio/kodoop
HADOOP_COMMAND: hadoop
SLIDER_COMMAND: /home/kognitio/kodoop/slider/bin/slider
KODOOP_LOGS: /home/kognitio/kodoop/logs
KODOOP_RUNTIME: /home/kognitio/kodoop/clusters
HDFS seems to be working OK
Slider's client seems to be working OK
Check finished.

But when i am giving create cluster command like

kodoop create_cluster MyTestKog

then its showing

'Slider failed to start'
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Tue Jan 30, 2018 11:20 am

Re: Slider not working

by markm » Thu Feb 07, 2019 12:29 pm

Hi Ashish,

Can you please answer the following questions:

What is the exact version of Hadoop you are using?

What platform is it on (AWS, Azure (HDInsight?), GCP)?

Was slider configured automatically or did you add it to the cluster?

Did you build the cluster for this test or is it a cluster used by other applications?

Was Kerberos enabled when you built the cluster or did you enable it afterwards?

Thanks,
Mark
Reply with quote Top
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Re: Slider not working

by Ashish79 » Fri Feb 08, 2019 9:25 am

markm wrote:
Thu Feb 07, 2019 12:29 pm
Hi Ashish,

Can you please answer the following questions:

What is the exact version of Hadoop you are using?

What platform is it on (AWS, Azure (HDInsight?), GCP)?

Was slider configured automatically or did you add it to the cluster?

Did you build the cluster for this test or is it a cluster used by other applications?

Was Kerberos enabled when you built the cluster or did you enable it afterwards?

Thanks,
Mark
Hadoop Version --3.1.0
Platform-Azure
As per the steps given i have configured slider by adding ip adress of nodes on which zookeeper is running in slider-client.xml. Apart from it no configurations done.
I did not build the cluster and yes other apps are also using the cluster.
Cluster was already kerberized.
Reply with quote Top
Contributor
Offline
Posts: 20
Joined: Thu May 23, 2013 5:11 pm

Re: Slider not working

by Simon.Darkin » Fri Feb 08, 2019 12:19 pm

Anish,

Can you provide the following please?

The output from the cluster launch command

The kodoop logs from the edge node

The yarn application log for the app if it got going at all or is hung

Thanks

Simon
Reply with quote Top
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Re: Slider not working

by Ashish79 » Mon Feb 11, 2019 10:12 am

Simon.Darkin wrote:
Fri Feb 08, 2019 12:19 pm
Anish,

Can you provide the following please?

The output from the cluster launch command

The kodoop logs from the edge node

The yarn application log for the app if it got going at all or is hung

Thanks

Simon

Hi Simon

Below are kodoop logs

Sorted by name
" Sort sequence: [\/]$,\<core\%(\.\d\+\)\=\>,\.h$,\.c$,\.cpp$,\~\=\*$,*,\.o$,\.obj$,\.info$,\.swp$,\.bak$,\~$
" Quick Help: <F1>:help -:go up dir D:delete R:rename s:sort-by x:exec
" ============================================================================
../ 0 Mon 11 Feb 2019 10:15:25 AM UTC
./ 0 Mon 11 Feb 2019 10:15:25 AM UTC
logs-kog10/ 0 Mon 11 Feb 2019 07:41:03 AM UTC
logs-kog11/ 0 Mon 11 Feb 2019 07:56:49 AM UTC
logs-kog3/ 0 Thu 07 Feb 2019 12:45:35 PM UTC
logs-kog5/ 0 Thu 07 Feb 2019 01:21:50 PM UTC
logs-kog6/ 0 Fri 08 Feb 2019 10:59:03 AM UTC
logs-kog8/ 0 Fri 08 Feb 2019 01:13:35 PM UTC
logs-kog9/ 0 Mon 11 Feb 2019 06:38:30 AM UTC
actions-kog10.20190211 35 Mon 11 Feb 2019 07:41:12 AM UTC
actions-kog11.20190211 35 Mon 11 Feb 2019 07:56:59 AM UTC
actions-kog3.20190207 34 Thu 07 Feb 2019 12:45:44 PM UTC
actions-kog5.20190207 34 Thu 07 Feb 2019 01:21:59 PM UTC
actions-kog6.20190208 34 Fri 08 Feb 2019 10:59:11 AM UTC
actions-kog8.20190208 34 Fri 08 Feb 2019 01:03:45 PM UTC
actions-kog9.20190211 34 Mon 11 Feb 2019 06:38:39 AM UTC
commands 432311 Mon 11 Feb 2019 07:57:20 AM UTC
local-install-kog10.log 720 Mon 11 Feb 2019 07:41:03 AM UTC
local-install-kog11.log 720 Mon 11 Feb 2019 07:56:49 AM UTC
local-install-kog3.log 717 Thu 07 Feb 2019 12:45:35 PM UTC
local-install-kog5.log 717 Thu 07 Feb 2019 01:21:50 PM UTC
local-install-kog6.log 717 Fri 08 Feb 2019 10:59:03 AM UTC
local-install-kog8.log 717 Fri 08 Feb 2019 01:03:36 PM UTC
local-install-kog9.log 717 Mon 11 Feb 2019 06:38:30 AM UTC
wcommands 3874 Mon 11 Feb 2019 07:56:32 AM UTC
wg-109969.out 540 Thu 07 Feb 2019 01:21:34 PM UTC
wg-116844.out 548 Mon 11 Feb 2019 06:31:48 AM UTC
wg-125223.out 541 Mon 11 Feb 2019 07:56:33 AM UTC
wg-126548.out 533 Fri 08 Feb 2019 10:47:15 AM UTC
wg-128288.out 545 Mon 11 Feb 2019 06:38:14 AM UTC
wg-19268.out 543 Fri 08 Feb 2019 10:58:47 AM UTC
wg-30805.out 547 Thu 07 Feb 2019 12:34:53 PM UTC
wg-34162.out 542 Thu 07 Feb 2019 12:36:15 PM UTC
wg-34654.out 551 Thu 07 Feb 2019 12:36:42 PM UTC
wg-47175.out 546 Thu 07 Feb 2019 12:45:19 PM UTC
wg-82278.out 542 Fri 08 Feb 2019 01:03:21 PM UTC
wg-96594.out 543 Thu 07 Feb 2019 01:15:02 PM UTC
wg-97395.out 539 Mon 11 Feb 2019 06:21:28 AM UTC
wg-98083.out 542 Mon 11 Feb 2019 07:40:47 AM UTC


Thanks
Ashish
Reply with quote Top
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Re: Slider not working

by Ashish79 » Mon Feb 11, 2019 11:01 am

Anyone help me with below questions

Q1. Can we integrate Kognitio with HDP version 3.0.1 or not ?
Q2. I am getting error "unkown application instance " after invoking slider. (Note: HDFS and Slider seems working fine after invoking 'kodoop testenv' command)
Q3: In ambari 2.6 version we can create instance of slider but i am using ambari 2.7.1.(trying to resolve above mentioned error). Is it possible to create slider instance in ambari 2.7.1 ?
Reply with quote Top
Contributor
Offline
Posts: 20
Joined: Thu May 23, 2013 5:11 pm

Re: Slider not working

by Simon.Darkin » Mon Feb 11, 2019 11:17 am

Ashish,

I have sent you an email with details of the Kognitio SFTP server. Can you transfer the logs to that server and let me know when they are there please?

Regards

Simon
Reply with quote Top
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Kognitio Support on Hortonworks HDP 3.0.1

by Ashish79 » Mon Feb 11, 2019 11:46 am

Q. Does kognitio support on Hortonworks HDP 3.0.1 as slider component was depreciated from HDP 3.0.0 ?
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Tue Jan 30, 2018 11:20 am

Re: Slider not working

by markm » Mon Feb 11, 2019 11:47 am

Hi Ashish,

Can you please confirm the exact version of Hadoop you want to run on - here you say 3.0.1 but your previous answer said you wanted to use 3.1.0. Also, what distribution are you using - is it HDInsight or was it installed from the standard HDP distribution?

We are trying to replicate your problem but we need to know exactly what Hadoop version you're trying to install Kognitio on.

Could please you send us the output from these commands run on the system you are trying to install Kognitio on.

hdp-select versions

hdp-select status

Regards,
Mark
Reply with quote Top
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Re: Slider not working

by Ashish79 » Mon Feb 11, 2019 11:50 am

Simon.Darkin wrote:
Mon Feb 11, 2019 11:17 am
Ashish,

I have sent you an email with details of the Kognitio SFTP server. Can you transfer the logs to that server and let me know when they are there please?

Regards

Simon
Hi Simon,

I appreciate your efforts . Kindly send the email at ashish.mathur@ril.com.

Thanks
Reply with quote Top
Contributor
Offline
User avatar
Posts: 19
Joined: Wed Jun 05, 2013 1:19 pm

Re: Kognitio Support on Hortonworks HDP 3.0.1

by andym » Mon Feb 11, 2019 12:54 pm

Ashish79 wrote:
Mon Feb 11, 2019 11:46 am
Q. Does kognitio support on Hortonworks HDP 3.0.1 as slider component was depreciated from HDP 3.0.0 ?
Yes, Kognitio supports working with Hortonworks >= 3.0. You can use the Kognitio provided slider instead of the one which used to ship with Hortonworks.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Re: Slider not working

by Ashish79 » Mon Feb 11, 2019 1:22 pm

markm wrote:
Mon Feb 11, 2019 11:47 am
Hi Ashish,

Can you please confirm the exact version of Hadoop you want to run on - here you say 3.0.1 but your previous answer said you wanted to use 3.1.0. Also, what distribution are you using - is it HDInsight or was it installed from the standard HDP distribution?

We are trying to replicate your problem but we need to know exactly what Hadoop version you're trying to install Kognitio on.

Could please you send us the output from these commands run on the system you are trying to install Kognitio on.

hdp-select versions

hdp-select status

Regards,
Mark

Hi Mark i am using HDP 3.0.1.0-187
Reply with quote Top
Contributor
Offline
Posts: 20
Joined: Thu May 23, 2013 5:11 pm

Re: Slider not working

by Simon.Darkin » Mon Feb 11, 2019 2:09 pm

Ashish,

The yarn logs aren't in the filesystem. You have to go 'yarn logs --applicationId application.....' where application.... is the ID of the application slider was trying to start. Usually this will be in the URL which kodoop puts out on failure. Otherwise it will be in the list of applications the resource manager displays.

It might be that there is no yarn application depending on how far you got, but if you do have yarn logs could you copy those to the SFTP server too please?

Regards

Simon
Reply with quote Top
Contributor
Offline
User avatar
Posts: 10
Joined: Tue Feb 05, 2019 12:56 pm

Re: Kognitio Support on Hortonworks HDP 3.0.1

by Ashish79 » Tue Feb 12, 2019 5:26 am

andym wrote:
Mon Feb 11, 2019 12:54 pm
Ashish79 wrote:
Mon Feb 11, 2019 11:46 am
Q. Does kognitio support on Hortonworks HDP 3.0.1 as slider component was depreciated from HDP 3.0.0 ?
Yes, Kognitio supports working with Hortonworks >= 3.0. You can use the Kognitio provided slider instead of the one which used to ship with Hortonworks.
Hi
I am using compiled version of slider from " tar -zvxf kodoop-extras.tar.gz" file. But i am getting unknown instance application error. Kognitio is able to connect with Resource manager but its showing 'unknown instance application' . I found few steps on how to create instance in ambari but slider can't be configured in HDP 3.0.0. as slider fuctionality is depreciated from this version. I am searching an alternative to create slider instance.

Thanks
Reply with quote Top
Contributor
Offline
User avatar
Posts: 19
Joined: Wed Jun 05, 2013 1:19 pm

Re: Kognitio Support on Hortonworks HDP 3.0.1

by andym » Thu Feb 14, 2019 2:31 pm

Ashish79 wrote:
Tue Feb 12, 2019 5:26 am
Hi
I am using compiled version of slider from " tar -zvxf kodoop-extras.tar.gz" file. But i am getting unknown instance application error. Kognitio is able to connect with Resource manager but its showing 'unknown instance application' . I found few steps on how to create instance in ambari but slider can't be configured in HDP 3.0.0. as slider fuctionality is depreciated from this version. I am searching an alternative to create slider instance.

Thanks
Hi

Kognitio's 'kodoop-extras' package contains a version of slider which can be used as an alternative to the one which used to be supplied by Hortonworks. However I have done some experiments based on your description of the problem above and have discovered that there are some Hortonworks configurations (particularly with Kerberos) which do not work properly with the version of slider we provide.

I have built an alternative version of the kodoop-extras package for Hortonworks 3. You can get this from the 'all downloads' page on our website or just download it from here:
kodoop-extras-4-hdp.tar.gz

You can untar this bundle over the top of the previous one or into a new user's directory and you should then be able to create Kognitio clusters on your cluster.

Running on a kerberised cluster also requires the use of a kerberos keytab. You can make a keytab file for your kerberos principal like this:

Code: Select all

$ ktutil 
ktutil:  add_entry -password -p kodoop.krb@ANDYHDP.DV -k 1 -e des3-cbc-sha1-kd
Password for kodoop.krb@ANDYHDP.DV: 
ktutil:  wkt ~/kodoop.krb.kt
(run kinit first and use your own kerberos principal).
This will generate a file called kodoop.krb.kt (name it however you like) which you then need to upload into kodoop with the command:

Code: Select all

kodoop keytab upload <principal> filename
for my example:

Code: Select all

kodoop keytab upload kodoop.krb@ANDYHDP.DV kodoop.krb.kt 
When creating clusters you need to supply the principal like this:

Code: Select all

KODOOP_KEYTAB=kodoop.krb@ANDYHDP.DV kodoop create_cluster ...
You will see the kerberos information in the cluster summary text. For my example:

Code: Select all

$ KODOOP_KEYTAB=kodoop.krb@ANDYHDP.DV USE_NODE_SPACE=45 kodoop create_cluster krbtest16
Kognitio Analytical Platform software for Hadoop ver80204s190207.
(c)Copyright Kognitio Ltd 2001-2018.

=================================================================
Cluster configuration for krbtest16
Containers:               1
Container memsize:        23552 Mb
Container vcores:         4

Internal storage limit:   100 Gb per store
Internal store count:     1

External gateway port:    6550
Kognitio server version:  ver80204s190207

Cluster will use 23 Gb of ram.
Cluster will use  up to 100 Gb of HDFS storage for internal data.

Data networks:            all
Management networks:      all
Edge to cluster networks: all
Using keytab:             kodoop.krb@ANDYHDP.DV  principal kodoop.krb@ANDYHDP.DV

Use nodes from queue:     <<default>>
Stats reporting:          Disabled
=================================================================
Hit ctrl-c to abort or enter to continue
At this point I can hit enter and everything runs to completion.

If you do this with a user which has previously run the other verison of slider, the Yarn regstry code might have created a zookeeper key named /registry/users/<your username>/services/org-apache-slider which prevents this from working because it has the wrong ACL on it. Either run the above using a fresh user or remove the zookeeper key in question. I did the latter using the zookeeper user:

Code: Select all

$ sudo su - zookeeper
Last login: Fri Feb  8 17:31:59 GMT 2019
[zookeeper@ip-10-2-1-239 ~]$ zookeeper-client 
Connecting to localhost:2181
2019-02-11 18:50:03,215 - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.6-78--1, built on 12/06/2018 12:30 GMT

...etc...

[zk: localhost:2181(CONNECTED) 4] ls /registry/users/kodoop.krb/services/org-apache-slider
[]
(check this is empty)
[zk: localhost:2181(CONNECTED) 2] rmr /registry/users/kodoop.krb/services/org-apache-slider   
(removes the key.  Slider will re-make with the correct ACL)
Please give this a try and let us know how you get on.
Reply with quote Top

Who is online

Users browsing this forum: MSN [Bot] and 1 guest

cron