Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> only one map or reduce job per time on one node

Copy link to this message
Re: only one map or reduce job per time on one node
Does no one have an idea if it is possible? In my example the heapsize of
the tasktracker is 2gb. The max map heap size is 1gb and the mac heap size
of the reduce process 1,5gb. If there are running 2 maps or just the reduce
job everything is fine. But if there are running 2 maps + 1 reduce process
I got a "Error: java.lang.OutOfMemoryError: Java heap space at" message.

kind regards
2013/11/5 John <[EMAIL PROTECTED]>

> Because my node swaps the memory if the 2 map slots + 1 reduce is occupied
> with my job. Sure I can minimize the max memory for the map/reduce process.
> I tried this already, but I got a out of memory exception if set the max
> heap size for the map/reduce process to low for my mr job.
> kind regards
> 2013/11/5 Vinod Kumar Vavilapalli <[EMAIL PROTECTED]>
>> Why do you want to do this?
>> +Vinod
>> On Nov 5, 2013, at 9:17 AM, John wrote:
>> Is it possible to force the jobtracker executing only 2 map jobs or 1
>> reduce job per time?
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.