Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Problem replacing existing Hive file with modified copy


Copy link to this message
-
RE: Problem replacing existing Hive file with modified copy
Gee thanks! That is great service.

Chuck
From: Bejoy Ks [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, July 25, 2012 12:04 PM
To: [EMAIL PROTECTED]
Subject: Re: Problem replacing existing Hive file with modified copy

The corresponding jira filed to track this bug is 'HIVE-3300' .
https://issues.apache.org/jira/browse/HIVE-3300

Regards
Bejoy KS

________________________________
From: Bejoy Ks <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Sent: Wednesday, July 25, 2012 9:28 PM
Subject: Re: Problem replacing existing Hive file with modified copy
Hi Connell

It looks like a bug in hive, I checked with hive 0.9 . If you are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.

But if we do the 'LOAD DATA INPATH'  for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.

I'll file a jira for this issue and update the same here.

Regards
Bejoy KS

________________________________
From: "Connell, Chuck" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Sent: Wednesday, July 25, 2012 8:51 PM
Subject: Problem replacing existing Hive file with modified copy

I created a Hive table that consists of two files, names1.txt and names2.txt. The table works correctly and answers all queries etc.

I want to REPLACE names2.txt with a modified version. I copied the new version of names2.txt to the /tmp/input folder within HDFS. Then I tried the command:

hive -e "load data inpath '/tmp/input/names2.txt' into table names"

I get an error: "Failed with exception null. Exception error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask"

Is this just a bug in this version (hadoop-hive-0.7.1+42.43-2.noarch, via CDH3 distro)?

Is my syntax correct to update a file in Hive?

TIA,
Chuck Connell
Nuance R&D Data Team
Burlington, MA
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB