Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Metrics for kafka broker


Copy link to this message
-
Re: Metrics for kafka broker
What I mean is that we can't use JConsole, etc.  from outside of EC2 into
the EC2 machine.  I'm aware one can use jmxtrans on the same box.

On Mon, Oct 1, 2012 at 12:50 PM, Matthew Rathbone <[EMAIL PROTECTED]>wrote:

> Hey Evan,
>
> Not sure what you mean there, we're using JMX in EC2, what issues have you
> seen? You should just be able to open a port and point jmxtrans at it.
>
> On Mon, Oct 1, 2012 at 12:08 PM, Evan chan <[EMAIL PROTECTED]> wrote:
>
> > Well the point is that JMX doesn't work in EC2, so having coda hale
> > metrics from Kafka directly would be a win.
> >
> > -Evan
> > Carry your candle, run to the darkness
> > Seek out the helpless, deceived and poor
> > Hold out your candle for all to see it
> > Take your candle, and go light your world
> >
> >
> >
> > On Oct 1, 2012, at 11:38 AM, John Wang <[EMAIL PROTECTED]> wrote:
> >
> > > For the metrics requested earlier in the thread, you can easily export
> > > those metrics in the application layer.
> > >
> > > -John
> > >
> > > On Mon, Oct 1, 2012 at 11:34 AM, Evan Chan <[EMAIL PROTECTED]> wrote:
> > >
> > >> Any plans to incorporate coda hale metrics into Kafka?  That would be
> > >> awesome.
> > >>
> > >> On Mon, Oct 1, 2012 at 11:26 AM, John Wang <[EMAIL PROTECTED]>
> wrote:
> > >>
> > >>> Take a look at: http://metrics.codahale.com/
> > >>> Used in production at Yammer and LinkedIn.
> > >>>
> > >>> -John
> > >>>
> > >>> On Mon, Oct 1, 2012 at 8:08 AM, Matthew Rathbone <
> > [EMAIL PROTECTED]
> > >>>> wrote:
> > >>>
> > >>>> We use JMXTrans, it's pretty easy to set up. Pro-tip: use the
> > >>> yml-to-config
> > >>>> generator included as a contrib script, it's much more concise.
> > >>>>
> > >>>> On Sun, Sep 30, 2012 at 11:03 PM, Evan chan <[EMAIL PROTECTED]> wrote:
> > >>>>
> > >>>>> Maybe you can use something like jmxtrans, or Jolokia (JMX over
> > >> http)?
> > >>>>>
> > >>>>> -Evan
> > >>>>> Carry your candle, run to the darkness
> > >>>>> Seek out the helpless, deceived and poor
> > >>>>> Hold out your candle for all to see it
> > >>>>> Take your candle, and go light your world
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Sep 30, 2012, at 8:38 PM, Rakesh Nair <[EMAIL PROTECTED]>
> > >> wrote:
> > >>>>>
> > >>>>>> Hi
> > >>>>>>
> > >>>>>> I am using kafka-0.7.1. Is there a way i can make the broker
> > >> publish
> > >>>> some
> > >>>>>> metrics periodically in the log files. Something like "Topic:abc,
> > >>>>>> MsgsConsumed=N, DataSpeed=n msg/sec, ..... "
> > >>>>>>
> > >>>>>> --
> > >>>>>> Regards
> > >>>>>> Rakesh Nair
> > >>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Matthew Rathbone
> > >>>> Foursquare | Software Engineer | Server Engineering Team
> > >>>> [EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
> > >>>> 4sq<http://foursquare.com/rathboma>
> > >>>>
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> --
> > >> *Evan Chan*
> > >> Senior Software Engineer |
> > >> [EMAIL PROTECTED] | (650) 996-4600
> > >> www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > >> @ooyala<http://www.twitter.com/ooyala>
> > >>
> >
>
>
>
> --
> Matthew Rathbone
> Foursquare | Software Engineer | Server Engineering Team
> [EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
> 4sq<http://foursquare.com/rathboma>
>

--
--
*Evan Chan*
Senior Software Engineer |
[EMAIL PROTECTED] | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>