|
|
-
I need help defending the use of Hadoop on a mixed use clusterSteve Lewis 2012-07-30, 15:32
Another lab has a cluster with about a thousand nodes. I have been using
eight of their nodes for some hadoop development. Recently my group was offered the use of the entire cluster at times. They added the provision that we could not run Hadoop or even have a Hadoop server using nodes from the entire cluster or a significant portion of the nodes (say half) because "Hadoop uses too many resources". I need some help with the resources used by Hadoop - especially in idle mode when no jobs are being run except JobTracker, Namenodes and whatever idle processes are running. I am building a case for upper management that the cluster can be shared without significantly interfering with other jobs. Any help would be welcome. -- Steven M. Lewis PhD 4221 105th Ave NE Kirkland, WA 98033 206-384-1340 (cell) Skype lordjoe_com |