-Re: Ganglia Metrics Reporter
Andrew Otto 2013-11-01, 18:08
It looks as though adding multicast support is more an issue with the Metrics code rather than with kafka-ganglia. The only way I could see to tell GangliaReporter to use multicast when sending messages was to pass it a GangliaMessageBuilder that constructed GangliaMessages using a proper MulticastSocket which sets the multicast TTL.
I attempted to do this like this:
However, GangliaMessageBuilder is not a public class, so I can't extend it as MulticastGangliaMessageBuilder.
I'm not sure what to do at this point. I'd rather use this library but, it looks like I might have to go with jmxtrans after all. Thoughts?
On Oct 29, 2013, at 2:38 PM, Maxime Brugidou <[EMAIL PROTECTED]> wrote:
> Hi Andrew, how do you plan to use metrics 3.0.1? The current Kafka 0.8
> version uses 2.2 AFAIK so this is going to require a new Kafka version.
> I don't really use multicast TTL muself but you are right that this should
> be configurable and I'll definitely accept pull request going in that
> direction. It's a tiny piece of code so it shouldn't be hard to dig it.
> Thanks for using this small plugin and let me know what you plan to do.
> On Oct 29, 2013 2:00 PM, "Andrew Otto" <[EMAIL PROTECTED]> wrote:
>> Hi Maxime,
>> I'm using this at the Wikimedia Foundation to send Kafka Broker metrics to
>> Ganglia. However, we use Ganglia in multicast mode. This mostly seems to
>> work with your code, but the ttl on the multicast packets gets set at 1.
>> We have sometimes have multiple levels of ganglia aggregators, and with
>> ttl=1, the multicast packets don't make it to the proper aggregators.
>> I'm looking into either forking or rewriting this library using Codahale
>> Metrics v 3.0.1, and supporting multicast more explicitly. Is this
>> something you could do better/faster than me, or should I proceed? :)
>> -Andrew Otto
>> (Thanks for writing this, btw!)
>> On Aug 22, 2013, at 11:42 AM, Maxime Brugidou <[EMAIL PROTECTED]>
>>> Hi all,
>>> Since I couldn't find any other way to publish kafka metrics to ganglia
>>> from kafka 0.8 (beta), I just published on github a super-simple ganglia
>>> metrics reporter for Kafka. It is configurable through the kafka config
>>> file and you can use it on the broker side and on your
>>> There is also a feature to exclude some metrics with a regex (useful if
>>> have many topics/partions).
>>> Here it is on github: https://github.com/criteo/kafka-ganglia
>>> Let me know if you have issues/questions. This is using metrics-ganglia
>>> 2.2.0 directly so it is not a ganglia plugin but rather a kafka "add-on".
>>> I don't know the proper way to distribute this yet for installation on
>>> brokers, it could also be part of some contrib kafka code.