Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hive multiple inserts


Copy link to this message
-
Re: hive multiple inserts
hi,

I'v tried use hive svn version, seems this bug still exists.

svn st -v

          896805   896744 namit        .
           896805   894292 namit        eclipse-templates
           896805   894292 namit        eclipse-templates/.classpath
           896805   765509 zshao
eclipse-templates/TestHive.launchtemplate
           896805   765509 zshao        eclipse-templates/TestMTQueries.l
           ..........

svn reversion 896805 ?

follows is the execute log.

hive> from
test
    > INSERT OVERWRITE LOCAL DIRECTORY '/home/stefdong/tmp/0' select * where
a = 1
    > INSERT OVERWRITE LOCAL DIRECTORY '/home/stefdong/tmp/1' select * where
a = 3;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201001071716_4691, Tracking URL http://abc.com:50030/jobdetails.jsp?jobid=job_201001071716_4691
Kill Command = hadoop job  -Dmapred.job.tracker=abc.com:9001 -kill
job_201001071716_4691
2010-01-08 14:14:55,442 Stage-2 map = 0%,  reduce = 0%
2010-01-08 14:15:00,643 Stage-2 map = 100%,  reduce = 0%
Ended Job = job_201001071716_4691
Copying data to local directory /home/stefdong/tmp/0
Copying data to local directory /home/stefdong/tmp/0
13 Rows loaded to /home/stefdong/tmp/0
9 Rows loaded to /home/stefdong/tmp/1
OK
Time taken: 9.409 seconds
thx.

2010/1/6 wd <[EMAIL PROTECTED]>

> hi,
>
> Single insert can extract data into '/tmp/out/1'.I even can see "xxx rows
> loaded to '/tmp/out/0', xxx rows loaded to '/tmp/out/1'...etc in multi
> inserts, but there is no data in fact.
>
> Havn't try svn revision, will try it today.thx.
>
> 2010/1/5 Zheng Shao <[EMAIL PROTECTED]>
>
> Looks like a bug.
>> What is the svn revision of Hive?
>>
>> Did you verify that single insert into '/tmp/out/1' produces non-empty
>> files?
>>
>> Zheng
>>
>> On Tue, Jan 5, 2010 at 12:51 AM, wd <[EMAIL PROTECTED]> wrote:
>> > In hive wiki:
>> >
>> > Hive extension (multiple inserts):
>> > FROM from_statement
>> > INSERT OVERWRITE [LOCAL] DIRECTORY directory1 select_statement1
>> >
>> > [INSERT OVERWRITE [LOCAL] DIRECTORY directory2 select_statement2] ...
>> >
>> > I'm try to use hive multi inserts to extract data from hive to local
>> disk.
>> > Follows is the hql
>> >
>> > from test_tbl
>> > INSERT OVERWRITE LOCAL DIRECTORY '/tmp/out/0' select select * where
>> id%10=0
>> > INSERT OVERWRITE LOCAL DIRECTORY '/tmp/out/1' select select * where
>> id%10=1
>> > INSERT OVERWRITE LOCAL DIRECTORY '/tmp/out/2' select select * where
>> id%10=2
>> >
>> > This hql can execute, but only /tmp/out/0 have datafile in it, other
>> > directories are empty. why this happen? bug?
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Yours,
>> Zheng
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB