Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - check namenode, jobtracker, datanodes and tasktracker status


Copy link to this message
-
check namenode, jobtracker, datanodes and tasktracker status
Marc Sturlese 2011-07-08, 17:49
Hey there,
I've written some scripts to check dfs disk space, number of datanodes,
number of tasktrackers, heap in use...
I'm with hadoop 0.20.2 and to do that I use the DFSClient and JobClient
APIs.
I do things like:

JobClient jc = new JobClient(socketJT, conf);
ClusterStatus clusterStatus = jc.getClusterStatus(true);
clusterStatus.getTaskTrackers();
...
jc.close();
DFSClient client = new DFSClient(socketNN, conf);
DatanodeInfo[] dni = client.datanodeReport(DatanodeReportType.ALL);
...
client.close();

FileSystem fs = FileSystem.get(new URI("hdfs://" + host + "/"), conf);
fs.getStatus().getCapacity();
...
fs.close();

It's is working well but I'm worried it could be harmful for the cluster to
run the script continuously (resource consumer). Is it alrite for example to
run it every 10 o 15 minutes? In case not, which is a good practice to
monitor the cluster?

Thanks in advance.
--
View this message in context: http://lucene.472066.n3.nabble.com/check-namenode-jobtracker-datanodes-and-tasktracker-status-tp3152565p3152565.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.