Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Re: How to use comstom mapreduce script


Copy link to this message
-
Re: Re: How to use comstom mapreduce script
Hello Gang,

 I put together some examples and helper classes for writing hive
map/reduce scripts in java here:

http://dev.bizo.com/2009/10/hive-map-reduce-in-java.html

I hope it is useful.

thanks,
larry

2009/11/12 Larry Ogrodnek <[EMAIL PROTECTED]>:
> Hello Gang,
>
>  I put together some examples and helper classes for writing hive
> map/reduce scripts in java here:
>
> http://dev.bizo.com/2009/10/hive-map-reduce-in-java.html
>
> I hope it is useful.
>
> 2009/11/12 Gang Luo <[EMAIL PROTECTED]>:
>> Thanks Zheng. It helps.
>>
>>
>> -Gang
>>
>>
>>
>> ----- 原始邮件 ----
>> 发件人: Zheng Shao <[EMAIL PROTECTED]>
>> 收件人: [EMAIL PROTECTED]; [EMAIL PROTECTED]
>> 发送日期: 2009/11/12 (周四) 7:10:33 下午
>> 主   题: Re: How to use comstom mapreduce script
>>
>> Yes you can compile it into a jar, and insert the command line "java
>> /xxx/my.jar" into Hive queries.
>>
>> http://www.slideshare.net/ragho/hive-user-meeting-august-2009-facebook
>> Page 72 has an example
>>
>> IF your map/reduce function is simple, you can probably write a Hive
>> UDF instead.
>> In the near future, Hive will support UDTF and you can write a UDTF
>> for any map/reduce function. This will make it more efficient than the
>> current way (running "java /xxx/my.jar").
>>
>>
>> Zheng
>>
>> 2009/11/13 Gang Luo <[EMAIL PROTECTED]>:
>>> Hi all,
>>> It is said that we can plug our own map/reduce into hive. I got two
>>> questions here.
>>> 1) Can I write this 'script' in Java? If so, should I compile it and make it
>>> into a jar file first?
>>> 2) Should I load this script file into HDFS first?
>>>
>>> Thanks.
>>>
>>> --Gang
>>>
>>> ________________________________
>>> 好玩贺卡等你发,邮箱贺�
ㄈ�律舷撸�>>
>>
>>
>> --
>> Yours,
>> Zheng
>>
>>
>>
>>      ___________________________________________________________
>>  好玩贺卡等你发,邮箱贺�
ㄈ�律舷撸�>> http://card.mail.cn.yahoo.com/
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB