Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Re: Map JVMs not being terminated after MAP jobs complettion.


Copy link to this message
-
Re: Map JVMs not being terminated after MAP jobs complettion.
Rahul Bhattacharjee 2013-09-13, 16:45
After the map task finishes , the VM terminates and all resources are
reclaimable.

Reduces might start pulling data from completed map tasks much before all
the maps are finished. However there is a property using which you can
delay the reduce step till all the maps are completed.

Rahul
On Fri, Sep 13, 2013 at 12:42 PM, Manu Reddy <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am running a memory intensive task on my cluster. The reducer class
> should load more than 2gb into memory. I was running into memory issues as
> the Map JVMS were not terminated after the completion of map tasks and
> still were holding onto memory.
>
> Is this normal? What can I do to ensure that my reduce function gets
> called only after all the map jvms are down?
>
>
> --
> Manu Reddy,
> mobile no: 091-8143405225,
> Hyderabad.
>