Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely


Copy link to this message
-
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Did you try putting those values inside core-site.xml for hdfs?
On Tue, Sep 10, 2013 at 8:11 PM, <[EMAIL PROTECTED]> wrote:

>  Hi Nitin,****
>
> Latest update. Am able to start metastore and table getting created. But
> when I execute the command “hive> select * from table_name”, hive compaints
> ****
>
> ** **
>
> Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).****
>
> Time taken: 1.158 seconds****
>
> ** **
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 5:04 PM
> *To:* Haldar, Shouvanik; [EMAIL PROTECTED]
>
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
>  ** **
>
> Sorry again. File system will be hadoop
>
> Sent from my Windows Phone****
>   ------------------------------
>
> *From: *Haldar, Shouvanik <[EMAIL PROTECTED]>
> *Sent: *10-09-2013 16:59
> *To: *[EMAIL PROTECTED]
> *Subject: *RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> No. File system will be s3n, or native file system or hadoop. But using
> HIVE we are storing data that is already existing inside s3****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]<[EMAIL PROTECTED]>]
>
> *Sent:* Tuesday, September 10, 2013 4:56 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
>  ****
>
> Ok, so back to basics. ****
>
>  ****
>
> correct me If I am wrong****
>
>  ****
>
> You are trying to setup YARN using s3 as file system? ****
>
> Then you want to use hive on top of it. ****
>
>  ****
>
>  ****
>
> On Tue, Sep 10, 2013 at 4:48 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> The issue is if I paste it in core-site.xml of Hadoop on the node in
> question, then will starting datanode would do only? Actually,. I am a bit
> confused what all daemons to start. I have a YARN architecture running 1
> separate – Resource  Manager, 1 NameNode &  4 datanodes.****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Tuesday, September 10, 2013 4:43 PM****
>
>
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
>  ****
>
> There is not more to start. If hive is not starting properly, it will not
> start at all so you are doing ok. ****
>
>  ****
>
> Problem is when you create table, hive tries to create a directory with
> that name if its not existing. Just try to put those two settings inside
> core-site.xml and see if the hdfs can use it to create directory ****
>
>  ****
>
> On Tue, Sep 10, 2013 at 4:06 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> The values are showing when I run “hive> set;” command. But it’s not
> getting identified. I guess I am not starting HIVE properly. Do you know
> what are the steps?****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Tuesday, September 10, 2013 4:04 PM****
>
>
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
>  ****
>
> From the config file it looks ok. Not really sure why it is not picking up
> when you run it via hive cli. ****
>
>  ****
>
> Can you try putting these in core-site.xml of hdfs. ****
>
>  ****
>
>  ****
>
> <property>****
>
>     <name>fs.s3n.awsAccessKeyId</name>****
>
>     <value>ID</value>****
>
> </property>****
>
> <property>****
>
>     <name>fs.s3n.awsSecretAccessKey</name>****

Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB