Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - On a small cluster can we double up namenode/master with tasktrackers?


Copy link to this message
-
RE: On a small cluster can we double up namenode/master with tasktrackers?
David Parks 2013-03-20, 09:27
Good points all,

 

The mapreduce jobs are, well. intensive. We've got a whole variety, but
typically I see them use a lot of CPU, a lot of Disk, and upon occasion a
whole bunch of Network bandwidth.  Duh right?  J

 

The master node is mostly CPU intensive right? We're using LXC to segregate
(psudo-virtualize) our environments for ease of development and management.
I'm looking into whether I can use LXC's quota system to guarantee a certain
level of CPU resources to the container where the master node is housed. If
I can do that I guess we wouldn't have any issue here.

 

Thanks!

David

 

 

From: Jens Scheidtmann [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, March 19, 2013 3:12 PM
To: [EMAIL PROTECTED]
Subject: Re: On a small cluster can we double up namenode/master with
tasktrackers?

 

David,

You didn't look at how resource intensive your map/reduce jobs are.

 

Best regards,

Jens