Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Metrics for kafka broker


Copy link to this message
-
Re: Metrics for kafka broker
Ah, yeah we manage to do that using a VPN.

On Tue, Oct 2, 2012 at 12:30 PM, Patricio Echagüe <[EMAIL PROTECTED]>wrote:

> I'm using JConsole from outside EC2 to connect to our EC2 boxes. I'll be
> happy to share the script if you like.
>
> Pato
>
> On Tue, Oct 2, 2012 at 10:02 AM, Evan Chan <[EMAIL PROTECTED]> wrote:
>
> > What I mean is that we can't use JConsole, etc.  from outside of EC2 into
> > the EC2 machine.  I'm aware one can use jmxtrans on the same box.
> >
> > On Mon, Oct 1, 2012 at 12:50 PM, Matthew Rathbone <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > Hey Evan,
> > >
> > > Not sure what you mean there, we're using JMX in EC2, what issues have
> > you
> > > seen? You should just be able to open a port and point jmxtrans at it.
> > >
> > > On Mon, Oct 1, 2012 at 12:08 PM, Evan chan <[EMAIL PROTECTED]> wrote:
> > >
> > > > Well the point is that JMX doesn't work in EC2, so having coda hale
> > > > metrics from Kafka directly would be a win.
> > > >
> > > > -Evan
> > > > Carry your candle, run to the darkness
> > > > Seek out the helpless, deceived and poor
> > > > Hold out your candle for all to see it
> > > > Take your candle, and go light your world
> > > >
> > > >
> > > >
> > > > On Oct 1, 2012, at 11:38 AM, John Wang <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > For the metrics requested earlier in the thread, you can easily
> > export
> > > > > those metrics in the application layer.
> > > > >
> > > > > -John
> > > > >
> > > > > On Mon, Oct 1, 2012 at 11:34 AM, Evan Chan <[EMAIL PROTECTED]> wrote:
> > > > >
> > > > >> Any plans to incorporate coda hale metrics into Kafka?  That would
> > be
> > > > >> awesome.
> > > > >>
> > > > >> On Mon, Oct 1, 2012 at 11:26 AM, John Wang <[EMAIL PROTECTED]>
> > > wrote:
> > > > >>
> > > > >>> Take a look at: http://metrics.codahale.com/
> > > > >>> Used in production at Yammer and LinkedIn.
> > > > >>>
> > > > >>> -John
> > > > >>>
> > > > >>> On Mon, Oct 1, 2012 at 8:08 AM, Matthew Rathbone <
> > > > [EMAIL PROTECTED]
> > > > >>>> wrote:
> > > > >>>
> > > > >>>> We use JMXTrans, it's pretty easy to set up. Pro-tip: use the
> > > > >>> yml-to-config
> > > > >>>> generator included as a contrib script, it's much more concise.
> > > > >>>>
> > > > >>>> On Sun, Sep 30, 2012 at 11:03 PM, Evan chan <[EMAIL PROTECTED]>
> > wrote:
> > > > >>>>
> > > > >>>>> Maybe you can use something like jmxtrans, or Jolokia (JMX over
> > > > >> http)?
> > > > >>>>>
> > > > >>>>> -Evan
> > > > >>>>> Carry your candle, run to the darkness
> > > > >>>>> Seek out the helpless, deceived and poor
> > > > >>>>> Hold out your candle for all to see it
> > > > >>>>> Take your candle, and go light your world
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Sep 30, 2012, at 8:38 PM, Rakesh Nair <[EMAIL PROTECTED]>
> > > > >> wrote:
> > > > >>>>>
> > > > >>>>>> Hi
> > > > >>>>>>
> > > > >>>>>> I am using kafka-0.7.1. Is there a way i can make the broker
> > > > >> publish
> > > > >>>> some
> > > > >>>>>> metrics periodically in the log files. Something like
> > "Topic:abc,
> > > > >>>>>> MsgsConsumed=N, DataSpeed=n msg/sec, ..... "
> > > > >>>>>>
> > > > >>>>>> --
> > > > >>>>>> Regards
> > > > >>>>>> Rakesh Nair
> > > > >>>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> --
> > > > >>>> Matthew Rathbone
> > > > >>>> Foursquare | Software Engineer | Server Engineering Team
> > > > >>>> [EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma
> >
> > |
> > > > >>>> 4sq<http://foursquare.com/rathboma>
> > > > >>>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> --
> > > > >> *Evan Chan*
> > > > >> Senior Software Engineer |
> > > > >> [EMAIL PROTECTED] | (650) 996-4600
> > > > >> www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > > > >> @ooyala<http://www.twitter.com/ooyala>
> > > > >>
> > > >
> > >
> > >
> > >
> > > --
> > > Matthew Rathbone
> > > Foursquare | Software Engineer | Server Engineering Team
> > > [EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |

Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
[EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB