Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Multiple consumer connectors in process - lag metrics?


Copy link to this message
-
Re: Multiple consumer connectors in process - lag metrics?
Ah, that makes sense Jun.  By default, the clientid is the same as the
groupid?  But we can override that for individual connectors sharing the
same groupid, I see.

Thanks,

Jason
On Sat, Nov 16, 2013 at 2:18 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> You could use different client id in those consumers. Our metrics name for
> maxlag includes the client id.
>
> Thanks,
>
> Jun
>
>
> On Thu, Nov 14, 2013 at 7:47 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > We are experimenting with having multiple consumer connectors running in
> > the same process, under the same groupId (but with different topic
> > filters).
> >
> > I'm wondering what the expected effect of this is with metrics, like
> > ConsumerFetcherManager.<groupid>-MaxLag
> >
> > It looks like in AbstractFetcherManager, that it creates a yammer Gauge
> for
> > this, probably repeatedly for each connector created?
> >
> > I'm not sure this metric will do the right thing?  Will it just show the
> > first (or the last one) registered?
> >
> > Jason
> >
>