Forum

Information and discussion related to the Kognitio on Hadoop product
Single Poster
Offline
User avatar
Posts: 1
Joined: Wed Dec 06, 2017 12:46 pm

Failed to download: s3://elasticmapreduce/libs/script-runner/script-runner.jar

by vedprakash01 » Wed Dec 06, 2017 12:54 pm

I am getting below error when i am trying to create the cluster. I am following steps mentioned in the https://kognitio.com/blog/kognitio-on-emr/

2017-12-06T11:48:20.133Z INFO Ensure step 2 jar file s3://elasticmapreduce/libs/script-runner/script-runner.jar
INFO Failed to download: s3://elasticmapreduce/libs/script-runner/script-runner.jar
java.lang.RuntimeException: Error whilst fetching 's3://elasticmapreduce/libs/script-runner/script-runner.jar'
at aws157.instancecontroller.util.S3Wrapper.fetchS3HadoopFileToLocal(S3Wrapper.java:412)
at aws157.instancecontroller.util.S3Wrapper.fetchHadoopFileToLocal(S3Wrapper.java:351)
at aws157.instancecontroller.master.steprunner.HadoopJarStepRunner$Runner.<init>(HadoopJarStepRunner.java:243)
at aws157.instancecontroller.master.steprunner.HadoopJarStepRunner.createRunner(HadoopJarStepRunner.java:152)
at aws157.instancecontroller.master.steprunner.HadoopJarStepRunner.createRunner(HadoopJarStepRunner.java:146)
at aws157.instancecontroller.master.steprunner.StepExecutor.runStep(StepExecutor.java:118)
at aws157.instancecontroller.master.steprunner.StepExecutor.run(StepExecutor.java:69)
at aws157.instancecontroller.master.steprunner.StepExecutionManager.enqueueStep(StepExecutionManager.java:248)
at aws157.instancecontroller.master.steprunner.StepExecutionManager.doRun(StepExecutionManager.java:195)
at aws157.instancecontroller.master.steprunner.StepExecutionManager.access$000(StepExecutionManager.java:33)
at aws157.instancecontroller.master.steprunner.StepExecutionManager$1.run(StepExecutionManager.java:94)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. (Service: Amazon S3; Status Code: 301; Error Code: PermanentRedirect; Request ID: 21E5FFE4A9F20498), S3 Extended Request ID: tGT6Elt2JqjVEz42rFhw+KELOJUzD3YgU3rvD1hO1YbVibquxbXknB/mBvl0sVCTv2aT1ZJ/E9k=
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1389)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:902)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:607)
at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:376)
at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:338)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:287)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3826)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1143)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1021)
at aws157.instancecontroller.util.S3Wrapper.copyS3ObjectToFile(S3Wrapper.java:303)
at aws157.instancecontroller.util.S3Wrapper.getFile(S3Wrapper.java:287)
at aws157.instancecontroller.util.S3Wrapper.fetchS3HadoopFileToLocal(S3Wrapper.java:399)
... 10 more
Reply with quote Top
Contributor
Offline
Posts: 38
Joined: Mon Jan 06, 2014 10:36 am

Re: Failed to download: s3://elasticmapreduce/libs/script-runner/script-runner.jar

by skkirkham » Thu Dec 07, 2017 9:38 am

Hi

This error suggests that the path you have entered for the AWS EMR script is incorrect. More details on this jar file can be found here

http://docs.aws.amazon.com/emr/latest/R ... cript.html

You could try something like

s3://<region>.elasticmapreduce/libs/script-runner/script-runner.jar

where <region> is the AWS region where your emr nodes will be located.

As an alternative if you want to install direct onto a cluster that already exists (i.e. spin up your emr cluster first) then install Kognitio on Hadoop afterwards there are step by step instructions in the documentation here:

https://kognitio.com/documentation/late ... odoop.html

Make sure your emr cluster has YARN, HDFS, and ZooKeeper installed when you spin it up and you will have all the pre-requisites covered.
Let me know how you get on

Regards
Sharon
Reply with quote Top

Who is online

Users browsing this forum: No registered users and 1 guest

cron