Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely


Copy link to this message
-
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Nitin Pawar 2013-09-11, 20:24
Did you try putting those values inside core-site.xml for hdfs?
On Tue, Sep 10, 2013 at 8:11 PM, <[EMAIL PROTECTED]> wrote:

>  Hi Nitin,****
>
> Latest update. Am able to start metastore and table getting created. But
> when I execute the command “hive> select * from table_name”, hive compaints
> ****
>
> ** **
>
> Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).****
>
> Time taken: 1.158 seconds****
>
> ** **
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 5:04 PM
> *To:* Haldar, Shouvanik; [EMAIL PROTECTED]
>
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
>  ** **
>
> Sorry again. File system will be hadoop
>
> Sent from my Windows Phone****
>   ------------------------------
>
> *From: *Haldar, Shouvanik <[EMAIL PROTECTED]>
> *Sent: *10-09-2013 16:59
> *To: *[EMAIL PROTECTED]
> *Subject: *RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> No. File system will be s3n, or native file system or hadoop. But using
> HIVE we are storing data that is already existing inside s3****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]<[EMAIL PROTECTED]>]
>
> *Sent:* Tuesday, September 10, 2013 4:56 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
>  ****
>
> Ok, so back to basics. ****
>
>  ****
>
> correct me If I am wrong****
>
>  ****
>
> You are trying to setup YARN using s3 as file system? ****
>
> Then you want to use hive on top of it. ****
>
>  ****
>
>  ****
>
> On Tue, Sep 10, 2013 at 4:48 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> The issue is if I paste it in core-site.xml of Hadoop on the node in
> question, then will starting datanode would do only? Actually,. I am a bit
> confused what all daemons to start. I have a YARN architecture running 1
> separate – Resource  Manager, 1 NameNode &  4 datanodes.****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Tuesday, September 10, 2013 4:43 PM****
>
>
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
>  ****
>
> There is not more to start. If hive is not starting properly, it will not
> start at all so you are doing ok. ****
>
>  ****
>
> Problem is when you create table, hive tries to create a directory with
> that name if its not existing. Just try to put those two settings inside
> core-site.xml and see if the hdfs can use it to create directory ****
>
>  ****
>
> On Tue, Sep 10, 2013 at 4:06 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> The values are showing when I run “hive> set;” command. But it’s not
> getting identified. I guess I am not starting HIVE properly. Do you know
> what are the steps?****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Tuesday, September 10, 2013 4:04 PM****
>
>
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
>  ****
>
> From the config file it looks ok. Not really sure why it is not picking up
> when you run it via hive cli. ****
>
>  ****
>
> Can you try putting these in core-site.xml of hdfs. ****
>
>  ****
>
>  ****
>
> <property>****
>
>     <name>fs.s3n.awsAccessKeyId</name>****
>
>     <value>ID</value>****
>
> </property>****
>
> <property>****
>
>     <name>fs.s3n.awsSecretAccessKey</name>****

Nitin Pawar