Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka Metrics in 0.8


Copy link to this message
-
Re: Kafka Metrics in 0.8
Does the Kafka data show up in the log dir in kafka-2? Since only the
leader can serve any traffic, if there is only have one partition or all
leaders are on one broker, not all metrics (e.g., produce/fetch rate) will
show up on both brokers.

Thanks,

Jun
On Mon, Jul 29, 2013 at 6:04 AM, Hanish Bansal <
[EMAIL PROTECTED]> wrote:

> Hi All,
>
> As kafka-0.8's main feature in data replication.
>
> We are running two kafka nodes(lets say: kafka-1 and kafka-2) in cluster
> mode. If we produce data into kafka using replication factor=2 then its
> replica is created on second machine.
>
> I am using jconsole to monitor kafka. if messages are going on kafka-1 then
> i am able to see kafka metrics on jconsole of kafka-1. But by connecting to
> jconsole for kafka-2 i am not seeing any metrics, i want to know why it is
> happening ??
>
> For more clarity i am asking same question in other way:
> If one kafka node is storing replicas of data, then will it be visible on
> jconsole or not ?
>
>
>
>
>
>
> On Wed, Jul 3, 2013 at 1:06 PM, Hanish Bansal <
> [EMAIL PROTECTED]> wrote:
>
> > Thanks Jun
> >
> >
> > On Tue, Jul 2, 2013 at 2:35 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> >> In 0.8, there are more mbeans than 0.7.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >>
> >> On Mon, Jul 1, 2013 at 9:06 PM, Hanish Bansal <
> >> [EMAIL PROTECTED]> wrote:
> >>
> >> > Okay i'll try for same.
> >> >
> >> > Also want to know that in kafka-0.7 there is single MBean -
> >> > SocketServerStats which provides all kafka Metrics. But in Kafka-0-8
> >> there
> >> > are individual MBeans (like AllTopicsBytesInPerSec,
> >> AllTopicsBytesOutPerSec
> >> > etc.) for getting each parameter like ByteInRate, ByteOutRate etc.
> >> >
> >> > Does kafka-0.8 provide any single MBean from which we can get all
> >> > parameters?
> >> >
> >> >
> >> >
> >> >
> >> > On Mon, Jul 1, 2013 at 3:47 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >> >
> >> > > It seems that the mbean name that you used is wrong. The mbean names
> >> > > registered by metrics 2.2.0 have quotes in them.
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > >
> >> > > On Mon, Jul 1, 2013 at 4:31 AM, Hanish Bansal <
> >> > > [EMAIL PROTECTED]> wrote:
> >> > >
> >> > > > Hi
> >> > > >
> >> > > > I am getting various kafka parameters (like NumFetchRequests,
> >> > > > FetchRequestsPerSecond, ProduceRequestsPerSecond etc.) through JMX
> >> in
> >> > > case
> >> > > > of kafka-0.7.
> >> > > >
> >> > > > In kafka-0.8 i am doing the same but not able to get parameters.
> >> > > >
> >> > > > There are some changes in reporting metrics in kafka-0.8 as
> >> compared to
> >> > > > kafka-0.7. I have already noticed that. According to changes in
> >> > > kafka-0.8 i
> >> > > > am trying to get kafka metrics using JMX through jmxtrans api.
> >> > > >
> >> > > > Here is sample java code of getting jmx parameters using jmxtrans
> >> api:
> >> > > >
> >> > > >        Server kafkaJMXServer = new Server("127.0.0.1", "9999");
> >> > > >         Query query1 = new Query();
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> query1.setObj("kafka.server:type=BrokerTopicMetrics,name=AllTopicsBytesInPerSec");
> >> > > >         query1.addAttr("MeanRate");
> >> > > >         kafkaJMXServer.addQuery(query1);
> >> > > >         JmxProcess process = new JmxProcess(kafkaJMXServer);
> >> > > >         JmxUtils.execute(process);
> >> > > >         System.out.println(query1.getResults());
> >> > > >
> >> > > > Please let me know what i am missing?
> >> > > >
> >> > > > --
> >> > > > *Thanks & Regards*
> >> > > > *Hanish Bansal*
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > *Thanks & Regards*
> >> > *Hanish Bansal*
> >> >
> >>
> >
> >
> >
> > --
> > *Thanks & Regards*
> > *Hanish Bansal*
> >
>
>
>
> --
> *Thanks & Regards*
> *Hanish Bansal*
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB