Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HIVE and S3 via EMR?


Copy link to this message
-
Re: HIVE and S3 via EMR?
Currently EMR only supports Hive versions 0.7.x AFAIK.

Russell, you may have to use Florin's suggestion – however, since your table is not partitioned, you will have to use something like "alter table set location". Note that this will change the location of your Hive table from its default location to your location in S3. If that is not what you want, you will have to physically copy it down to HDFS/file system and then do the load.

Sriram

From: Ashutosh Chauhan <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tue, 29 May 2012 13:24:38 -0700
To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: HIVE and S3 via EMR?

Which hive version you are using? You need fix of https://issues.apache.org/jira/browse/HIVE-1444 which was released in 0.9.0

Thanks,
Ashutosh

On Tue, May 29, 2012 at 1:20 PM, Russell Jurney <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
How do I load data from S3 into Hive using Amazon EMR?  I've booted a small cluster, and I want to load a 3-column TSV file from Pig into a table like this:

create table from_to (from_address string, to_address string, dt string);

When I run something like this:

load data inpath 's3n://rjurney_public_web/from_to_date' into table from_to;

I get errors:

FAILED: Error in semantic analysis: Line 1:17 Invalid path 's3n://rjurney_public_web/from_to_date': only "file" or "hdfs" file systems accepted. s3n file system is not supported.

There is no distcp on the master node of my EMR cluster, so I can't copy it over.  I've read the documentation... and so far after a day of trying, I can't load data into HIVE via EMR.

What am I missing?  Thanks!
--
Russell Jurney twitter.com/rjurney<http://twitter.com/rjurney> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> datasyndrome.com<http://datasyndrome.com/>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB