Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - JMXTrans not sending kafka 0.8 metrics to Ganglia


Copy link to this message
-
Re: JMXTrans not sending kafka 0.8 metrics to Ganglia
Priya Matpadi 2013-11-05, 06:00
Paul,
Thank you! I didn't realize the quotes are important to keep.
I have question on name=\"*\""

For the type=ReplicaManager, there are at least 4 Mbean names
"kafka.server":name="PartitionCount",type="ReplicaManager"
"kafka.server":name="LeaderCount",type="ReplicaManager"

"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
"kafka.server":name="ISRExpandsPerSec",type="ReplicaManager"

In ganglia metrics dropdown, I only see 4 elements as configured above i.e.

ReplicaManager.Count
ReplicaManager.OneMinuteRate
ReplicaManager.MeanRate
ReplicaManager.Value

Then, how can one tell if for e.g. Value pertains to LeaderCount or
PartitionCount?
Even though it is laborious, isn't it good idea to explicitly specify the
name?

Thanks,
Priya
On Mon, Nov 4, 2013 at 5:10 PM, Paul Mackles <[EMAIL PROTECTED]> wrote:

> It looks like you are missing quotes in the object name. Here is a snippet
> from our jmxtrans configs:
>
> "resultAlias": "ReplicaManager",
>      "obj": "\"kafka.server\":type=\"ReplicaManager\",name=\"*\"",
>      "attr": [
>       "Count",
>       "OneMinuteRate",
>       "MeanRate",
>       "Value"
>      ]
>
>
> Unless more recent versions of kafka get rid of the quotes (we are on an
> older 0.8 version still).
>
> Paul
>
> On 11/4/13 8:02 PM, "Priya Matpadi" <[EMAIL PROTECTED]> wrote:
>
> >I am trying to send kafka metrics for display to ganglia server using
> >latest download from https://github.com/adambarthelson/kafka-ganglia.
> >
> >Here's my kafka_metrics.json
> >{
> >  "servers" : [ {
> >    "port" : "9999",
> >    "host" : "ecokaf1",
> >    "queries" : [ {
> >      "outputWriters" : [ {
> >        "@class" :
> >"com.googlecode.jmxtrans.model.output.GangliaWriter",
> >        "settings" : {
> >          "groupName" : "kafka stats 2",
> >          "port" : 8649,
> >          "host" : "ecokaf1"
> >        }
> >      } ],
> >      "obj" : "kafka.server:type=ReplicaManager,name=PartitionCount",
> >      "resultAlias": "Kafka.ReplicaManager",
> >      "attr" : [ "Value" ]
> >    } ],
> >    "numQueryThreads" : 2
> >  } ]
> >}
> >
> >I start jmxtrans as follows:
> >sudo ./jmxtrans.sh start kafka_metrics.json
> >
> >
> >JMXTrans starts without any issues. Here are the logs:
> >
> >[05 Nov 2013 00:49:48] [main] 0      INFO
> >(com.googlecode.jmxtrans.JmxTransformer:134) - Starting Jmxtrans on :
> >kafka_metrics.json
> >[05 Nov 2013 00:49:48] [main] 386    DEBUG
> >(com.googlecode.jmxtrans.JmxTransformer:354) - Loaded file:
> >/usr/share/jmxtrans/kafka_metrics.json
> >[05 Nov 2013 00:49:48] [main] 392    DEBUG
> >(com.googlecode.jmxtrans.model.output.GangliaWriter:119) - Validated
> >Ganglia metric [host: ecokaf1, port: 8649, addressingMode: UNICAST, ttl:
> >5,
> >v3.1: true, units: '', slope: BOTH, tmax: 60, dmax: 0, spoofedHostName:
> >192.168.3.1:ecokaf1, groupName: 'kafka stats 2']
> >[05 Nov 2013 00:49:48] [main] 406    DEBUG
> >(com.googlecode.jmxtrans.JmxTransformer:429) - Scheduled job:
> >ecokaf1:9999-1383612588479-3580845919 for server: Server [host=ecokaf1,
> >port=9999, url=null, cronExpression=null, numQueryThreads=2]
> >[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 414    DEBUG
> >(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
> >Server [host=ecokaf1, port=9999, url=null, cronExpression=null,
> >numQueryThreads=2]
> >[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 544    DEBUG
> >(com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 1 query
> >threads
> >[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 553    DEBUG
> >(com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
> >Server [host=ecokaf1, port=9999,
> >url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi,
> >cronExpression=null,
> >numQueryThreads=2]
> >[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60401  DEBUG
> >(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
> >Server [host=ecokaf1, port=9999,
> >url=service:jmx:rmi:///jndi/rmi://ecokaf1