Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Metrics for kafka broker


Copy link to this message
-
Re: Metrics for kafka broker
Ah, yeah we manage to do that using a VPN.

On Tue, Oct 2, 2012 at 12:30 PM, Patricio Echagüe <[EMAIL PROTECTED]>wrote:

> I'm using JConsole from outside EC2 to connect to our EC2 boxes. I'll be
> happy to share the script if you like.
>
> Pato
>
> On Tue, Oct 2, 2012 at 10:02 AM, Evan Chan <[EMAIL PROTECTED]> wrote:
>
> > What I mean is that we can't use JConsole, etc.  from outside of EC2 into
> > the EC2 machine.  I'm aware one can use jmxtrans on the same box.
> >
> > On Mon, Oct 1, 2012 at 12:50 PM, Matthew Rathbone <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > Hey Evan,
> > >
> > > Not sure what you mean there, we're using JMX in EC2, what issues have
> > you
> > > seen? You should just be able to open a port and point jmxtrans at it.
> > >
> > > On Mon, Oct 1, 2012 at 12:08 PM, Evan chan <[EMAIL PROTECTED]> wrote:
> > >
> > > > Well the point is that JMX doesn't work in EC2, so having coda hale
> > > > metrics from Kafka directly would be a win.
> > > >
> > > > -Evan
> > > > Carry your candle, run to the darkness
> > > > Seek out the helpless, deceived and poor
> > > > Hold out your candle for all to see it
> > > > Take your candle, and go light your world
> > > >
> > > >
> > > >
> > > > On Oct 1, 2012, at 11:38 AM, John Wang <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > For the metrics requested earlier in the thread, you can easily
> > export
> > > > > those metrics in the application layer.
> > > > >
> > > > > -John
> > > > >
> > > > > On Mon, Oct 1, 2012 at 11:34 AM, Evan Chan <[EMAIL PROTECTED]> wrote:
> > > > >
> > > > >> Any plans to incorporate coda hale metrics into Kafka?  That would
> > be
> > > > >> awesome.
> > > > >>
> > > > >> On Mon, Oct 1, 2012 at 11:26 AM, John Wang <[EMAIL PROTECTED]>
> > > wrote:
> > > > >>
> > > > >>> Take a look at: http://metrics.codahale.com/
> > > > >>> Used in production at Yammer and LinkedIn.
> > > > >>>
> > > > >>> -John
> > > > >>>
> > > > >>> On Mon, Oct 1, 2012 at 8:08 AM, Matthew Rathbone <
> > > > [EMAIL PROTECTED]
> > > > >>>> wrote:
> > > > >>>
> > > > >>>> We use JMXTrans, it's pretty easy to set up. Pro-tip: use the
> > > > >>> yml-to-config
> > > > >>>> generator included as a contrib script, it's much more concise.
> > > > >>>>
> > > > >>>> On Sun, Sep 30, 2012 at 11:03 PM, Evan chan <[EMAIL PROTECTED]>
> > wrote:
> > > > >>>>
> > > > >>>>> Maybe you can use something like jmxtrans, or Jolokia (JMX over
> > > > >> http)?
> > > > >>>>>
> > > > >>>>> -Evan
> > > > >>>>> Carry your candle, run to the darkness
> > > > >>>>> Seek out the helpless, deceived and poor
> > > > >>>>> Hold out your candle for all to see it
> > > > >>>>> Take your candle, and go light your world
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Sep 30, 2012, at 8:38 PM, Rakesh Nair <[EMAIL PROTECTED]>
> > > > >> wrote:
> > > > >>>>>
> > > > >>>>>> Hi
> > > > >>>>>>
> > > > >>>>>> I am using kafka-0.7.1. Is there a way i can make the broker
> > > > >> publish
> > > > >>>> some
> > > > >>>>>> metrics periodically in the log files. Something like
> > "Topic:abc,
> > > > >>>>>> MsgsConsumed=N, DataSpeed=n msg/sec, ..... "
> > > > >>>>>>
> > > > >>>>>> --
> > > > >>>>>> Regards
> > > > >>>>>> Rakesh Nair
> > > > >>>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> --
> > > > >>>> Matthew Rathbone
> > > > >>>> Foursquare | Software Engineer | Server Engineering Team
> > > > >>>> [EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma
> >
> > |
> > > > >>>> 4sq<http://foursquare.com/rathboma>
> > > > >>>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> --
> > > > >> *Evan Chan*
> > > > >> Senior Software Engineer |
> > > > >> [EMAIL PROTECTED] | (650) 996-4600
> > > > >> www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > > > >> @ooyala<http://www.twitter.com/ooyala>
> > > > >>
> > > >
> > >
> > >
> > >
> > > --
> > > Matthew Rathbone
> > > Foursquare | Software Engineer | Server Engineering Team
> > > [EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |

Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
[EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>