In this case if you are not using INSERT INTO, you may need some tmp table write the query output to that. Load that data from there to your target table's data dir.
You are not writing that to any file while doing the LOAD DATA operation. Rather you are just moving the files(in hdfs) from the source location to the table's data dir (where the previous data files are present). In hdfs move operation there is just a meta data operation happening at file system level.
Go with INSERT INTO as it is a cleaner way in hql perspective.
Bejoy K S
From handheld, Please excuse typos.
From: Bhavesh Shah <[EMAIL PROTECTED]>
Date: Wed, 15 Feb 2012 15:03:07
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Doubt in INSERT query in Hive?
Hi Bejoy K S,
Thanks for your reply.
The overhead is, in select query I have near about 85 columns. Writing this
in the file and again loading it may take some time.
For that reason I am thinking that it will be inefficient.
On Wed, Feb 15, 2012 at 2:51 PM, <[EMAIL PROTECTED]> wrote:
> Hi Bhavesh
> INSERT INTO is supported in hive 0.8 . An upgrade would get you things
> LOAD DATA inefficient? What was the performance overhead you were facing
> Bejoy K S
> From handheld, Please excuse typos.
> *From: * Bhavesh Shah <[EMAIL PROTECTED]>
> *Date: *Wed, 15 Feb 2012 14:33:29 +0530
> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Doubt in INSERT query in Hive?
> Whenever we want to insert into table we use:
> INSERT OVERWRITE TABLE TBL_NAME
> (SELECT ....)
> Due to this, table gets overwrites everytime.
> I don't want to overwrite table, I want append it everytime.
> I thought about LOAD TABLE , but writing the file may take more time and I
> don't think so that it will efficient.
> Does Hive Support INSERT INTO TABLE TAB_NAME?
> (I am using hive-0.7.1)
> Is there any patch for it? (But I don't know How to apply patch ?)
> Pls suggest me as soon as possible.
> Bhavesh Shah