Forum

Information and discussion related to the Kognitio on Hadoop product
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

unable to query the external tables

by kveeraga » Tue Apr 28, 2020 3:50 pm

HY000 [Kognitio][WX2 Driver][<host>:<port>] ET0102: The external table connector for the operation was not found

Info:


Action
Attachments
kognitio_errors.PNG
kognitio_errors.PNG (23.84 KiB) Viewed 89440 times
Reply with quote Top
Contributor
Offline
Posts: 39
Joined: Mon Jan 06, 2014 10:36 am

Re: unable to query the external tables

by skkirkham » Tue Apr 28, 2020 5:52 pm

Hi kveeraga,

Sorry to hear you are having trouble with Kognitio external tables. Can you provide some more details:
1) What version of Kognitio on Hadoop are you running?
2) What SQL command are you submitting?
3) Can you also run EXPLAIN TABLE <myschema>.<mytablename> and supply the results of this query.

Many thanks
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Tue Apr 28, 2020 9:51 pm

I am running
we followed this post which worked previously for Tabby. viewtopic.php?f=13&t=522 now we both tried and it did not work
so now I tried to restart the cluster and server both and do an image after.
please find my console output screen shot I attached what I am running into

HY000: [Kognitio][WX2 Driver][unix://tmp/container_e158_1586671753320_110986_01_000016/wxdb-<system_id>/odbc] DS0800: Error encountered - repair required.
--> Create image failed. Setting login mode to sys-only.
Error running crimage, rc 65280.
Failed, rc 0x40000001.
Last edited by kveeraga on Wed Apr 29, 2020 10:16 pm, edited 1 time in total.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Tue Apr 28, 2020 9:58 pm

skkirkham wrote:
Tue Apr 28, 2020 5:52 pm
Hi kveeraga,

Sorry to hear you are having trouble with Kognitio external tables. Can you provide some more details:
1) What version of Kognitio on Hadoop are you running?
2) What SQL command are you submitting?
3) Can you also run EXPLAIN TABLE <myschema>.<mytablename> and supply the results of this query.

Many thanks
HDP 2.6.5

System ID <system_id>
Version 80200
Patch rel170824
Nodes 24
Cores 1,152
Disk GB 1,749.5
RAM GB 2,082.5
RAID none
Compilers 16
Interpreters 33
Up Time 46m
Ram Stores 600

2. sql command: select * from dev.t_transaction_cubs_2018;
3. Create text:-
CREATE EXTERNAL TABLE "DEV"."T_TRANSACTION_CUBS_2018" ("IDTRANSACTION" BIGINT, "DATE" DATE, "LOCATION_ID" INTEGER, "RTLR_LANE" SMALLINT, "TIME" TIME(0), "BASKET_ID" VARCHAR(255) CHARACTER SET LATI
N1, "CONSUMER_ID_ID" BIGINT, "AMOUNT" DECIMAL(10,4), "RTLR_TRANS_ID" BIGINT, "RTLR_LOCATION" INTEGER, "RTLR_DATE" TIMESTAMP(6), "RTLR_CASHIER" INTEGER, "ITEM_COUNT" SMALLINT, "INCENTIVE_COUNT" TIN
YINT, "TAX_AMOUNT" DECIMAL(10,4), "TENDER_TYPE_NAME" VARCHAR(255) CHARACTER SET LATIN1, "CREATED_BY" VARCHAR(255) CHARACTER SET LATIN1, "CREATED_SOURCE" VARCHAR(255) CHARACTER SET LATIN1, "CREATED
_TIMESTAMP" TIMESTAMP(6), "UPDATED_SOURCE" VARCHAR(255) CHARACTER SET LATIN1, "UPDATED_BY" VARCHAR(255) CHARACTER SET LATIN1, "UPDATED_TIMESTAMP" TIMESTAMP(6), "TRANSACTION_STATUS" VARCHAR(255) CH
ARACTER SET LATIN1, "CLIENTID" SMALLINT, "TRANSYEAR" SMALLINT, "TRANSMONTH" TINYINT) FROM "HDFS_ORC" TARGET 'uri_path "/apps/hive/warehouse/mdot_client.db/transaction/clientid=180/transyear=2018/t
ransmonth={10,11,12}/*"'
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Wed Apr 29, 2020 3:14 pm

Thanks for the detailed info. The 'DS0800' error can sometimes indicate corrupted data on disk. If this is the case then a repairing process will have to be run to patch out the corrupted areas on disk. Do you have a recent backup for this system? Please run the following steps, which may take some time to complete:

1. login as sys
2. run 'lock system;'. This is intended to prevent other users from logging in. If this errors due to other users having already logged in, then run 'update ipe_cursessions set abort=2 where session <> current_session;', followed by 'lock system;'.
3. Once the 'lock system' is successful, then run 'invoke disk_check;'
4. When the disk check is complete, make sure you have a <system_id> shell by running 'kodoop mgr <system_id> shell' in Linux, then tar up the `wxlogd smd` directory (note the backquotes). This tarball should then be attached here or emailed to helpdesk@kognitio.com.

Once we have the tarball we will investigate and let you know which tables are affected.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Wed Apr 29, 2020 5:55 pm

michaeld wrote:
Wed Apr 29, 2020 3:14 pm
Thanks for the detailed info. The 'DS0800' error can sometimes indicate corrupted data on disk. If this is the case then a repairing process will have to be run to patch out the corrupted areas on disk. Do you have a recent backup for this system? Please run the following steps, which may take some time to complete:

1. login as sys
2. run 'lock system;'. This is intended to prevent other users from logging in. If this errors due to other users having already logged in, then run 'update ipe_cursessions set abort=2 where session <> current_session;', followed by 'lock system;'.
3. Once the 'lock system' is successful, then run 'invoke disk_check;'
4. When the disk check is complete, make sure you have a <system_id> shell by running 'kodoop mgr <system_id> shell' in Linux, then tar up the `wxlogd smd` directory (note the backquotes). This tarball should then be attached here or emailed to helpdesk@kognitio.com.

Once we have the tarball we will investigate and let you know which tables are affected.
I ran the invoke disk check, the query is pending
Last edited by kveeraga on Wed Apr 29, 2020 10:17 pm, edited 1 time in total.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Wed Apr 29, 2020 5:58 pm

You won't be able to run it in Console, as Console always keeps more than 1 session active. It needs to be run using a command line utility, after locking the system as described below. You can connect by running 'kodoop sql <system_id>' on the edge node for example.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Wed Apr 29, 2020 6:05 pm

michaeld wrote:
Wed Apr 29, 2020 5:58 pm
You won't be able to run it in Console, as Console always keeps more than 1 session active. It needs to be run using a command line utility, after locking the system as described below. You can connect by running 'kodoop sql <system_id>' on the edge node for example.
I entered the sys login password or is there a default password for this login?
Last edited by kveeraga on Wed Apr 29, 2020 10:17 pm, edited 1 time in total.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Wed Apr 29, 2020 6:14 pm

"sessions inhibited" indicates that the login failed because another login already has a system lock (an exclusive connection, blocking other logins). Did you already run 'lock system' in console? If so, is the 'invoke disk_check' currently running?
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Wed Apr 29, 2020 6:28 pm

michaeld wrote:
Wed Apr 29, 2020 6:14 pm
"sessions inhibited" indicates that the login failed because another login already has a system lock (an exclusive connection, blocking other logins). Did you already run 'lock system' in console? If so, is the 'invoke disk_check' currently running?
I cant tell if it is running this is what I see
Last edited by kveeraga on Wed Apr 29, 2020 10:17 pm, edited 1 time in total.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Wed Apr 29, 2020 6:31 pm

Please disconnect from Console, and see if you can then login from command line.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Wed Apr 29, 2020 7:47 pm

michaeld wrote:
Wed Apr 29, 2020 6:31 pm
Please disconnect from Console, and see if you can then login from command line.
I disconnected from the console and I was able to login with the sys password do I now run the invoke disk_check;?
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Wed Apr 29, 2020 7:49 pm

Please lock the system now, then run the disk_check.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Thu Apr 30, 2020 4:30 pm

michaeld wrote:
Wed Apr 29, 2020 7:49 pm
Please lock the system now, then run the disk_check.
I took more than 5 hours
<user2>@<edge node>:~ $ kodoop sql <system_id>
Kognitio Analytical Platform software for Hadoop ver80200rel170824.
(c)Copyright Kognitio Ltd 2001-2017.

Kognitio WX2 SQL Submission Tool v8.02.00-rel170824
(c)Copyright Kognitio Ltd 1992-2017.

Password:
Connected to localhost:<port> ODBC Version 8.02.00-rel170824 Server Version 08.02.0000
>lock system;
Query 1 Complete ---- 0:00.1 0:00.1 0:00.1
>invoke disk_check;

Query 2 Complete ---- 5:26:48 5:26:48 5:26:48
>+
I sent the tar file to helpdesk
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Fri May 01, 2020 3:58 pm

kveeraga wrote:
Thu Apr 30, 2020 4:30 pm
michaeld wrote:
Wed Apr 29, 2020 7:49 pm
Please lock the system now, then run the disk_check.
I took more than 5 hours
<user2>@<edge node>:~ $ kodoop sql <system_id>
Kognitio Analytical Platform software for Hadoop ver80200rel170824.
(c)Copyright Kognitio Ltd 2001-2017.

Kognitio WX2 SQL Submission Tool v8.02.00-rel170824
(c)Copyright Kognitio Ltd 1992-2017.

Password:
Connected to localhost:<port> ODBC Version 8.02.00-rel170824 Server Version 08.02.0000
>lock system;
Query 1 Complete ---- 0:00.1 0:00.1 0:00.1
>invoke disk_check;

Query 2 Complete ---- 5:26:48 5:26:48 5:26:48
>+
I sent the tar file to helpdesk
Hi any update on the log review
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Fri May 01, 2020 6:33 pm

After reviewing the logs, there is some data corruption. It's impossible to say how many records have been lost, but the following table_ids have lost 1 or more records:

31761,
31764,
31771,
31805,
31845,
32157,
32457,
534,
539,
540,
541

You'll need to run a disk_repair now:

1. login as sys
2. run 'lock system;'. This is intended to prevent other users from logging in. If this errors due to other users having already logged in, then run 'update ipe_cursessions set abort=2 where session <> current_session;', followed by 'lock system;'.
3. Once the 'lock system' is successful, then run 'invoke disk_repair;'

Once the repair is complete, you should be able to restart with a system image and then reimage the system. Some records will have been lost from the above tables, so you'll need to decide whether this can be tolerated or whether to restore the tables from backup. This won't be necessary for the logging tables (534,539,540,541).

All virtual disks are missing a block of records in slab 6 (a user slab) and another in slab 2 (the logging slab). The missing piece is before the end of the disk, implying that there was a period of time during which data was not being correctly written out to HDFS and that this was later fixed. Since this is an old version of Kodoop (a 2017 release) it may be the case that HDFS went into safemode and Kognitio didn't deal with this properly. There have been improvements in this area, so we would recommend that after all the above steps have been run, you schedule an upgrade to the latest 8.2.0 version, available at https://kognitio.com/all-downloads/.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Tue May 05, 2020 1:20 am

michaeld wrote:
Fri May 01, 2020 6:33 pm
After reviewing the logs, there is some data corruption. It's impossible to say how many records have been lost, but the following table_ids have lost 1 or more records:

31761,
31764,
31771,
31805,
31845,
32157,
32457,
534,
539,
540,
541

You'll need to run a disk_repair now:

1. login as sys
2. run 'lock system;'. This is intended to prevent other users from logging in. If this errors due to other users having already logged in, then run 'update ipe_cursessions set abort=2 where session <> current_session;', followed by 'lock system;'.
3. Once the 'lock system' is successful, then run 'invoke disk_repair;'

Once the repair is complete, you should be able to restart with a system image and then reimage the system. Some records will have been lost from the above tables, so you'll need to decide whether this can be tolerated or whether to restore the tables from backup. This won't be necessary for the logging tables (534,539,540,541).

All virtual disks are missing a block of records in slab 6 (a user slab) and another in slab 2 (the logging slab). The missing piece is before the end of the disk, implying that there was a period of time during which data was not being correctly written out to HDFS and that this was later fixed. Since this is an old version of Kodoop (a 2017 release) it may be the case that HDFS went into safemode and Kognitio didn't deal with this properly. There have been improvements in this area, so we would recommend that after all the above steps have been run, you schedule an upgrade to the latest 8.2.0 version, available at https://kognitio.com/all-downloads/.
kodoop2@pedge1:~ $ kodoop sql <system_id>
Kognitio Analytical Platform software for Hadoop ver80200rel170824.
(c)Copyright Kognitio Ltd 2001-2017.

Kognitio WX2 SQL Submission Tool v8.02.00-rel170824
(c)Copyright Kognitio Ltd 1992-2017.

Password:
Connected to localhost:portno ODBC Version 8.02.00-rel170824 Server Version 08.02.0000
>lock system;
Query 1 Complete ---- 0:00.7 0:00.7 0:00.7
>invoke disk_repair;
Query 2 retcode = -1 ---- 7:21:30 7:21:30 7:21:30
08S02: [Kognitio][WX2 Driver] Session has been aborted
08S01: [Kognitio][WX2 Driver] Communications link failure
>
I tried twice and both time I got the same message
I sent the logs to the email
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Wed May 06, 2020 2:20 pm

Have you sent the log directory to helpdesk@kognitio.com yet? I haven't seen it arrive.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Wed May 06, 2020 3:49 pm

michaeld wrote:
Wed May 06, 2020 2:20 pm
Have you sent the log directory to helpdesk@kognitio.com yet? I haven't seen it arrive.
I sent them again today please check
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Wed May 06, 2020 5:01 pm

Thanks, we've received the logs. Unfortunately there's been a server crash during the disk repair. To investigate this we will need a dump of the process that crashed. Has the server been restarted since 2020-05-04 21:37:13 GMT? If not, please run the following steps:

1. on the edge node, run 'kodoop mgr <system_id> shell'
2. run 'wxprobe -s', and send us the full output.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Wed May 06, 2020 6:28 pm

michaeld wrote:
Wed May 06, 2020 5:01 pm
Thanks, we've received the logs. Unfortunately there's been a server crash during the disk repair. To investigate this we will need a dump of the process that crashed. Has the server been restarted since 2020-05-04 21:37:13 GMT? If not, please run the following steps:

1. on the edge node, run 'kodoop mgr <system_id> shell'
2. run 'wxprobe -s', and send us the full output.
no it is hasnt been started yet. I have sent the out from wxprobe -s to the email
Reply with quote Top
Contributor
Offline
User avatar
Posts: 26
Joined: Mon Dec 01, 2014 11:38 am

Re: unable to query the external tables

by michaeld » Wed May 06, 2020 6:41 pm

Thanks. Now please do the following:

1. on the edge node, run 'kodoop mgr <system id> shell'
2. cd to a suitable directory where a ~2GB file can be saved. Run 'wxdgdump -m 471 -C ds471.core'
3. run 'wxdgtool -D -O .'
4. create a compressed tarball of the ds471.core file with 'tar -czvf ds471.tar.gz ds471.core'
5. send us the ds471.tar.gz file, as well as the history file and info file that was created in step 3. If the ds471.tar.gz file is too big to email, let us know and we'll arrange another transfer method.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 13
Joined: Fri Apr 24, 2020 10:52 pm

Re: unable to query the external tables

by kveeraga » Wed May 06, 2020 10:12 pm

michaeld wrote:
Wed May 06, 2020 6:41 pm
Thanks. Now please do the following:

1. on the edge node, run 'kodoop mgr <system id> shell'
2. cd to a suitable directory where a ~2GB file can be saved. Run 'wxdgdump -m 471 -C ds471.core'
3. run 'wxdgtool -D -O .'
4. create a compressed tarball of the ds471.core file with 'tar -czvf ds471.tar.gz ds471.core'
5. send us the ds471.tar.gz file, as well as the history file and info file that was created in step 3. If the ds471.tar.gz file is too big to email, let us know and we'll arrange another transfer method.
sent history file to the email however I do need a transfer method for ds471.tar.gz
Reply with quote Top

Who is online

Users browsing this forum: No registered users and 0 guests

cron