Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> When is configure and close run


Copy link to this message
-
Re: When is configure and close run
Thank you! Just to confirm. Consider a JVM (that is being reused), has
to reduce K1,{V11,V12,V13..} and K2,{V21,V22,V23,....}. Then the
configure and close methods are called once each for both K1,{V11,...}
and K2,{V2,....}?

Is my understanding correct?

Once again, there is no combiner, and it makes sense that it is not called.

Thank you
Saptarshi
On Mon, Jun 22, 2009 at 10:55 PM, jason hadoop<[EMAIL PROTECTED]> wrote:
> configure and close are run for each task, mapper and reducer. The configure
> and close are NOT run on the combiner class.
>
> On Mon, Jun 22, 2009 at 9:23 AM, Saptarshi Guha <[EMAIL PROTECTED]>
> wrote:
>>
>> Hello,
>> In a mapreduce job, a given map JVM will run N map tasks. Are the
>> configure and close methods executed for every one of these N tasks?
>> Or is configure executed once when the JVM starts and the close method
>> executed once when all N have been completed?
>>
>> I have the same question for the reduce task. Will it be run before
>> for every reduce task? And close is run when all the values for a
>> given key have been processed?
>>
>> We can assume there isn't a combiner.
>>
>> Regards
>> Saptarshi
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB