Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Re: How to use comstom mapreduce script


Copy link to this message
-
Re: Re: How to use comstom mapreduce script
Larry Ogrodnek 2009-11-13, 01:18
Hello Gang,

 I put together some examples and helper classes for writing hive
map/reduce scripts in java here:

http://dev.bizo.com/2009/10/hive-map-reduce-in-java.html

I hope it is useful.

thanks,
larry

2009/11/12 Larry Ogrodnek <[EMAIL PROTECTED]>:
> Hello Gang,
>
>  I put together some examples and helper classes for writing hive
> map/reduce scripts in java here:
>
> http://dev.bizo.com/2009/10/hive-map-reduce-in-java.html
>
> I hope it is useful.
>
> 2009/11/12 Gang Luo <[EMAIL PROTECTED]>:
>> Thanks Zheng. It helps.
>>
>>
>> -Gang
>>
>>
>>
>> ----- 原始邮件 ----
>> 发件人: Zheng Shao <[EMAIL PROTECTED]>
>> 收件人: [EMAIL PROTECTED]; [EMAIL PROTECTED]
>> 发送日期: 2009/11/12 (周四) 7:10:33 下午
>> 主   题: Re: How to use comstom mapreduce script
>>
>> Yes you can compile it into a jar, and insert the command line "java
>> /xxx/my.jar" into Hive queries.
>>
>> http://www.slideshare.net/ragho/hive-user-meeting-august-2009-facebook
>> Page 72 has an example
>>
>> IF your map/reduce function is simple, you can probably write a Hive
>> UDF instead.
>> In the near future, Hive will support UDTF and you can write a UDTF
>> for any map/reduce function. This will make it more efficient than the
>> current way (running "java /xxx/my.jar").
>>
>>
>> Zheng
>>
>> 2009/11/13 Gang Luo <[EMAIL PROTECTED]>:
>>> Hi all,
>>> It is said that we can plug our own map/reduce into hive. I got two
>>> questions here.
>>> 1) Can I write this 'script' in Java? If so, should I compile it and make it
>>> into a jar file first?
>>> 2) Should I load this script file into HDFS first?
>>>
>>> Thanks.
>>>
>>> --Gang
>>>
>>> ________________________________
>>> 好玩贺卡等你发,邮箱贺�
ㄈ�律舷撸�>>
>>
>>
>> --
>> Yours,
>> Zheng
>>
>>
>>
>>      ___________________________________________________________
>>  好玩贺卡等你发,邮箱贺�
ㄈ�律舷撸�>> http://card.mail.cn.yahoo.com/
>>
>