-Re: Simplify chukwa agent
Bill Graham 2010-10-18, 15:21
+1 on refactoring the agent code and it's related interfaces at some
point. Implementing an adaptor seems a bit more confusing than it
ought to be. Some of the methods on Adaptor, like getCurrentStatus for
instance, do quite a bit more than their javadoc implies, or the
method name would lead you to believe.
On Sun, Oct 17, 2010 at 11:14 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> Hi Ari,
> The list of TODO from my side for Chukwa 0.5 is:
> - Update demux parsers (New parser for parsing hadoop metrics, job
> history 0.20+ log, and hadoop logs)
> - Update data analytics scripts using pig 0.8
> - Update documentation
> I am waiting on pig to release 0.8 which contains support for using
> HBase as storage. This will enable chukwa data analytics on top of
> After pig 0.8 is released, then Chukwa 0.5 release will be ready. I
> agree that refactoring the interface in Chukwa Agent could wait until
> Chukwa 0.5 is released.
> On Sun, Oct 17, 2010 at 10:54 PM, Ariel Rabkin <[EMAIL PROTECTED]> wrote:
>> I agree that that part of the code is pretty snarly. Not sure how
>> urgent a priority it is to fix.
>> I think a better short-term goal might be doing a 0.5 release. How
>> close are we to that?
>> On Sun, Oct 17, 2010 at 3:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>> Hi all,
>>> Chukwa agent code is not intuitive to understand. This is mainly the
>>> interface in Chukwa agent is over complicating the implementation. I
>>> don't see a rationale for having each class to be an interface.
>>> Connector and Chukwa Sender are two interfaces which are not very
>>> useful to be interface. It creates over complicated subsystem to
>>> maintain collector list in agent, connector and sender. Ideally,
>>> there should be a single place for configuration source of truth. I
>>> am leaning toward making those interface abstract classes. Connector
>>> should be rename to something more meaningful like multiplexer or MUX
>>> for short. AgentControlSocketListener could be refactor into a jersey
>>> like rest api, for easier to maintain the code base, and remove the
>>> 9093 protocol.
>>> This sounds like a major task. I like to gather feedback to see if
>>> this change is necessary for creating more traction on chukwa
>> Ari Rabkin [EMAIL PROTECTED]
>> UC Berkeley Computer Science Department