-I need help defending the use of Hadoop on a mixed use cluster
Steve Lewis 2012-07-30, 15:32
Another lab has a cluster with about a thousand nodes. I have been using
eight of their nodes for some hadoop development. Recently my group was
offered the use of the entire cluster at times. They added the provision
that we could not run Hadoop or even have a Hadoop server using nodes from
the entire cluster or a significant portion of the nodes (say half) because
"Hadoop uses too many resources".
I need some help with the resources used by Hadoop - especially in idle
mode when no jobs are being run except JobTracker, Namenodes and whatever
idle processes are running. I am building a case for upper management that
the cluster can be shared without significantly interfering with other jobs.
Any help would be welcome.
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033