Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> On a small cluster can we double up namenode/master with tasktrackers?


Copy link to this message
-
RE: On a small cluster can we double up namenode/master with tasktrackers?
Good points all,

 

The mapreduce jobs are, well. intensive. We've got a whole variety, but
typically I see them use a lot of CPU, a lot of Disk, and upon occasion a
whole bunch of Network bandwidth.  Duh right?  J

 

The master node is mostly CPU intensive right? We're using LXC to segregate
(psudo-virtualize) our environments for ease of development and management.
I'm looking into whether I can use LXC's quota system to guarantee a certain
level of CPU resources to the container where the master node is housed. If
I can do that I guess we wouldn't have any issue here.

 

Thanks!

David

 

 

From: Jens Scheidtmann [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, March 19, 2013 3:12 PM
To: [EMAIL PROTECTED]
Subject: Re: On a small cluster can we double up namenode/master with
tasktrackers?

 

David,

You didn't look at how resource intensive your map/reduce jobs are.

 

Best regards,

Jens

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB