Forum

Information and discussion related to the Kognitio on Hadoop product
Contributor
Offline
User avatar
Posts: 15
Joined: Mon Jun 04, 2018 4:28 pm

How to create HDFS_ORC connector?

by tajones » Wed Sep 19, 2018 9:25 pm

We have a kognitio cluster on hadoop that does not have the HDFS_ORC connector set up (I did not do the original installation, but assumed this should have been installed by default). I tried the following to set this up but it's not working when I try to select from the table that consists of orc files. I was trying to mimic the setup we have in another kognitio cluster but also reference the documentation but since they are not aligned I might be missing something.

Cluster Info: ver80200rel170616

Originally I copied /opt/kognitio/java-ext-table-bin-0-4-8.tar.gz and extracted it from our working cluster and placed it on all nodes (all in /opt/kognitio)

I ran this (naming it ORC_LOCAL instead):
create connector ORC_LOCAL
command '/opt/kognitio/java-ext-table/connector'
target ' uri_location "hdfs://NONPRODHADOOP", hadoop_native_lib "/usr/hdp/2.6.5.0-292/hadoop/lib/native/", connector_type ORC, ram_mb_required 500';

I created my table and referenced "FROM "ORC_LOCAL" TARGET 'uri_path "/path/to/orc/files"';"

Tried to select data and received this error:
HY000 [Kognitio][WX2 Driver][kognitiodev] ET0003: Error calling an external table connector method
Info:
An external table connector returned an error handling a method call.
Action:
check the external table connector's handler script

When this didn't work, I tried to copy java-ext-table.plugin from our working cluster and placed it here on the edge node: /home/kodoop/kodoop/add_on

Ran this as sys:
create connector HDFS_ORC
command 'java-ext-table.plugin'
target ' connector_mode PLUGIN, connector_type ORC, ram_mb_required 500';

Created my table using "FROM "HDFS_ORC" TARGET 'uri_path "/path/to/orc/files"'"

Selected from the table and received this error:
HY000 [Kognitio][WX2 Driver][kognitiodev] ET0003: Error calling an external table connector method
Info:
An external table connector returned an error handling a method call.
Action:
check the external table connector's handler script

To try to further debug I ran these two queries but there was no data:
select * from sys.ipe_script_debug where tno = (select max(tno) from sys.ipe_script_debug) order by pid, seq;
select cast(record as varchar(32000)), * from sys.ipe_conv_error where tno = (select max(tno) from sys.ipe_conv_error);

Can you provide some direction on what steps I might be missing? Thanks.
Reply with quote Top
Contributor
Offline
User avatar
Posts: 12
Joined: Tue Jan 30, 2018 11:20 am

Re: How to create HDFS_ORC connector?

by markm » Thu Sep 20, 2018 1:42 pm

Hi tajones,

The installation into /opt/kognitio method has been superseded by the plugin architecture and I don't think it was ever released as a free to use product.

You have the correct create connector statement for HDFS_ORC but the version of Kognitio you are using predates the plugin architecture.

We recommend that you use the latest Kognitio version which is available on https://kognitio.com/all-downloads/ - this will install the HDFS_ORC connector as standard.

Regards,
Mark
Reply with quote Top
Contributor
Offline
User avatar
Posts: 15
Joined: Mon Jun 04, 2018 4:28 pm

Re: How to create HDFS_ORC connector?

by tajones » Thu Sep 20, 2018 1:48 pm

Ok, thank you. I thought about that after I posted. I'll upgrade. Thanks!
Reply with quote Top

Who is online

Users browsing this forum: No registered users and 1 guest

cron