Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka Metrics in 0.8


+
Hanish Bansal 2013-07-01, 11:32
+
Jun Rao 2013-07-01, 15:48
+
Hanish Bansal 2013-07-02, 04:06
+
Jun Rao 2013-07-02, 14:35
+
Hanish Bansal 2013-07-03, 07:37
+
Hanish Bansal 2013-07-29, 13:05
Copy link to this message
-
Re: Kafka Metrics in 0.8
Does the Kafka data show up in the log dir in kafka-2? Since only the
leader can serve any traffic, if there is only have one partition or all
leaders are on one broker, not all metrics (e.g., produce/fetch rate) will
show up on both brokers.

Thanks,

Jun
On Mon, Jul 29, 2013 at 6:04 AM, Hanish Bansal <
[EMAIL PROTECTED]> wrote:

> Hi All,
>
> As kafka-0.8's main feature in data replication.
>
> We are running two kafka nodes(lets say: kafka-1 and kafka-2) in cluster
> mode. If we produce data into kafka using replication factor=2 then its
> replica is created on second machine.
>
> I am using jconsole to monitor kafka. if messages are going on kafka-1 then
> i am able to see kafka metrics on jconsole of kafka-1. But by connecting to
> jconsole for kafka-2 i am not seeing any metrics, i want to know why it is
> happening ??
>
> For more clarity i am asking same question in other way:
> If one kafka node is storing replicas of data, then will it be visible on
> jconsole or not ?
>
>
>
>
>
>
> On Wed, Jul 3, 2013 at 1:06 PM, Hanish Bansal <
> [EMAIL PROTECTED]> wrote:
>
> > Thanks Jun
> >
> >
> > On Tue, Jul 2, 2013 at 2:35 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> >> In 0.8, there are more mbeans than 0.7.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >>
> >> On Mon, Jul 1, 2013 at 9:06 PM, Hanish Bansal <
> >> [EMAIL PROTECTED]> wrote:
> >>
> >> > Okay i'll try for same.
> >> >
> >> > Also want to know that in kafka-0.7 there is single MBean -
> >> > SocketServerStats which provides all kafka Metrics. But in Kafka-0-8
> >> there
> >> > are individual MBeans (like AllTopicsBytesInPerSec,
> >> AllTopicsBytesOutPerSec
> >> > etc.) for getting each parameter like ByteInRate, ByteOutRate etc.
> >> >
> >> > Does kafka-0.8 provide any single MBean from which we can get all
> >> > parameters?
> >> >
> >> >
> >> >
> >> >
> >> > On Mon, Jul 1, 2013 at 3:47 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >> >
> >> > > It seems that the mbean name that you used is wrong. The mbean names
> >> > > registered by metrics 2.2.0 have quotes in them.
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > >
> >> > > On Mon, Jul 1, 2013 at 4:31 AM, Hanish Bansal <
> >> > > [EMAIL PROTECTED]> wrote:
> >> > >
> >> > > > Hi
> >> > > >
> >> > > > I am getting various kafka parameters (like NumFetchRequests,
> >> > > > FetchRequestsPerSecond, ProduceRequestsPerSecond etc.) through JMX
> >> in
> >> > > case
> >> > > > of kafka-0.7.
> >> > > >
> >> > > > In kafka-0.8 i am doing the same but not able to get parameters.
> >> > > >
> >> > > > There are some changes in reporting metrics in kafka-0.8 as
> >> compared to
> >> > > > kafka-0.7. I have already noticed that. According to changes in
> >> > > kafka-0.8 i
> >> > > > am trying to get kafka metrics using JMX through jmxtrans api.
> >> > > >
> >> > > > Here is sample java code of getting jmx parameters using jmxtrans
> >> api:
> >> > > >
> >> > > >        Server kafkaJMXServer = new Server("127.0.0.1", "9999");
> >> > > >         Query query1 = new Query();
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> query1.setObj("kafka.server:type=BrokerTopicMetrics,name=AllTopicsBytesInPerSec");
> >> > > >         query1.addAttr("MeanRate");
> >> > > >         kafkaJMXServer.addQuery(query1);
> >> > > >         JmxProcess process = new JmxProcess(kafkaJMXServer);
> >> > > >         JmxUtils.execute(process);
> >> > > >         System.out.println(query1.getResults());
> >> > > >
> >> > > > Please let me know what i am missing?
> >> > > >
> >> > > > --
> >> > > > *Thanks & Regards*
> >> > > > *Hanish Bansal*
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > *Thanks & Regards*
> >> > *Hanish Bansal*
> >> >
> >>
> >
> >
> >
> > --
> > *Thanks & Regards*
> > *Hanish Bansal*
> >
>
>
>
> --
> *Thanks & Regards*
> *Hanish Bansal*
>

 
+
Hanish Bansal 2013-07-30, 05:48
+
Jun Rao 2013-07-30, 16:04