Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - When is configure and close run


Copy link to this message
-
Re: When is configure and close run
Saptarshi Guha 2009-06-24, 17:00
Thank you! Just to confirm. Consider a JVM (that is being reused), has
to reduce K1,{V11,V12,V13..} and K2,{V21,V22,V23,....}. Then the
configure and close methods are called once each for both K1,{V11,...}
and K2,{V2,....}?

Is my understanding correct?

Once again, there is no combiner, and it makes sense that it is not called.

Thank you
Saptarshi
On Mon, Jun 22, 2009 at 10:55 PM, jason hadoop<[EMAIL PROTECTED]> wrote:
> configure and close are run for each task, mapper and reducer. The configure
> and close are NOT run on the combiner class.
>
> On Mon, Jun 22, 2009 at 9:23 AM, Saptarshi Guha <[EMAIL PROTECTED]>
> wrote:
>>
>> Hello,
>> In a mapreduce job, a given map JVM will run N map tasks. Are the
>> configure and close methods executed for every one of these N tasks?
>> Or is configure executed once when the JVM starts and the close method
>> executed once when all N have been completed?
>>
>> I have the same question for the reduce task. Will it be run before
>> for every reduce task? And close is run when all the values for a
>> given key have been processed?
>>
>> We can assume there isn't a combiner.
>>
>> Regards
>> Saptarshi
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>