Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Metrics ..


+
Mark Olimpiati 2012-08-29, 19:54
+
Wong, David 2012-08-29, 20:28
Hi David,

   I  enabled the "jvm.class" of the hadoop-metrics.properties, you're
output seems to be from something else (dfs.class or mapred.class) which
reports hadoop deamons performace. For example your output shows
"processName=TaskTracker"
which I'm not looking for.

  How can I report jvm statistics for individual jvms (maps/reducers) ??

Thank you,
Mark

On Wed, Aug 29, 2012 at 1:28 PM, Wong, David (DMITS) <[EMAIL PROTECTED]> wrote:

> Here's a snippet of tasktracker metrics using Metrics2.  (I think there
> were (more) gaps in the pre-metrics2 versions.)
> Note that you'll need to have hadoop-env.sh and hadoop-metrics2.properties
> setup on all the nodes you want reports from.
>
> 1345570905436 ugi.ugi: context=ugi, hostName=sqws31.caclab.cac.cpqcorp.net,
> loginSuccess_num_ops=0, loginSuccess_avg_time=0.0, loginFailure_num_ops=0,
> loginFailure_avg_time=0.0
> 1345570905436 jvm.metrics: context=jvm, processName=TaskTracker,
> sessionId=, hostName=sqws31.caclab.cac.cpqcorp.net,
> memNonHeapUsedM=11.540627, memNonHeapCommittedM=18.25,
> memHeapUsedM=12.972412, memHeapCommittedM=61.375, gcCount=1,
> gcTimeMillis=6, threadsNew=0, threadsRunnable=9, threadsBlocked=0,
> threadsWaiting=9, threadsTimedWaiting=1, threadsTerminated=0, logFatal=0,
> logError=0, logWarn=0, logInfo=1
> 1345570905436 mapred.tasktracker: context=mapred, sessionId=, hostName> sqws31.caclab.cac.cpqcorp.net, maps_running=0, reduces_running=0,
> mapTaskSlots=2, reduceTaskSlots=2, tasks_completed=0,
> tasks_failed_timeout=0, tasks_failed_ping=0
> 1345570905436 rpcdetailed.rpcdetailed: context=rpcdetailed, port=33997,
> hostName=sqws31.caclab.cac.cpqcorp.net
> 1345570905436 rpc.rpc: context=rpc, port=33997, hostName> sqws31.caclab.cac.cpqcorp.net, rpcAuthenticationSuccesses=0,
> rpcAuthenticationFailures=0, rpcAuthorizationSuccesses=0,
> rpcAuthorizationFailures=0, ReceivedBytes=0, SentBytes=0,
> RpcQueueTime_num_ops=0, RpcQueueTime_avg_time=0.0,
> RpcProcessingTime_num_ops=0, RpcProcessingTime_avg_time=0.0,
> NumOpenConnections=0, callQueueLen=0
> 1345570905436 metricssystem.MetricsSystem: context=metricssystem, hostName> sqws31.caclab.cac.cpqcorp.net, num_sources=5, num_sinks=1,
> sink.file.latency_num_ops=0, sink.file.latency_avg_time=0.0,
> sink.file.dropped=0, sink.file.qsize=0, snapshot_num_ops=5,
> snapshot_avg_time=0.2, snapshot_stdev_time=0.447213595499958,
> snapshot_imin_time=0.0, snapshot_imax_time=1.0, snapshot_min_time=0.0,
> snapshot_max_time=1.0, publish_num_ops=0, publish_avg_time=0.0,
> publish_stdev_time=0.0, publish_imin_time=3.4028234663852886E38,
> publish_imax_time=1.401298464324817E-45,
> publish_min_time=3.4028234663852886E38,
> publish_max_time=1.401298464324817E-45, dropped_pub_all=0
> 1345570915435 ugi.ugi: context=ugi, hostName=sqws31.caclab.cac.cpqcorp.net
> 1345570915435 jvm.metrics: context=jvm, processName=TaskTracker,
> sessionId=, hostName=sqws31.caclab.cac.cpqcorp.net,
> memNonHeapUsedM=11.549316, memNonHeapCommittedM=18.25,
> memHeapUsedM=13.136337, memHeapCommittedM=61.375, gcCount=1,
> gcTimeMillis=6, threadsNew=0, threadsRunnable=9, threadsBlocked=0,
> threadsWaiting=9, threadsTimedWaiting=1, threadsTerminated=0, logFatal=0,
> logError=0, logWarn=0, logInfo=1
> 1345570915435 mapred.tasktracker: context=mapred, sessionId=, hostName> sqws31.caclab.cac.cpqcorp.net, maps_running=0, reduces_running=0,
> mapTaskSlots=2, reduceTaskSlots=2
> 1345570915435 rpcdetailed.rpcdetailed: context=rpcdetailed, port=33997,
> hostName=sqws31.caclab.cac.cpqcorp.net
> 1345570915435 rpc.rpc: context=rpc, port=33997, hostName> sqws31.caclab.cac.cpqcorp.net
> 1345570915435 metricssystem.MetricsSystem: context=metricssystem, hostName> sqws31.caclab.cac.cpqcorp.net, num_sources=5, num_sinks=1,
> sink.file.latency_num_ops=1, sink.file.latency_avg_time=4.0,
> snapshot_num_ops=11, snapshot_avg_time=0.16666666666666669,
> snapshot_stdev_time=0.408248290463863, snapshot_imin_time=0.0,
> snapshot_imax_time=1.0, snapshot_min_time=0.0, snapshot_max_time=1.0,
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB