Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa, mail # dev - Simplify chukwa agent


Copy link to this message
-
Re: Simplify chukwa agent
Bill Graham 2010-10-18, 15:21
+1 on refactoring the agent code and it's related interfaces at some
point. Implementing an adaptor seems a bit more confusing than it
ought to be. Some of the methods on Adaptor, like getCurrentStatus for
instance, do quite a bit more than their javadoc implies, or the
method name would lead you to believe.
On Sun, Oct 17, 2010 at 11:14 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> Hi Ari,
>
> The list of TODO from my side for Chukwa 0.5 is:
>
> - Update demux parsers (New parser for parsing hadoop metrics, job
> history 0.20+ log, and hadoop logs)
> - Update data analytics scripts using pig 0.8
> - Update documentation
>
> I am waiting on pig to release 0.8 which contains support for using
> HBase as storage.  This will enable chukwa data analytics on top of
> pig+HBase.
> After pig 0.8 is released, then Chukwa 0.5 release will be ready.  I
> agree that refactoring the interface in Chukwa Agent could wait until
> Chukwa 0.5 is released.
>
> regards,
> Eric
>
> On Sun, Oct 17, 2010 at 10:54 PM, Ariel Rabkin <[EMAIL PROTECTED]> wrote:
>> Howdy.
>>
>> I agree that that part of the code is pretty snarly. Not sure how
>> urgent a priority it is to fix.
>>
>> I think a better short-term goal might be doing a 0.5 release. How
>> close are we to that?
>>
>> --Ari
>>
>> On Sun, Oct 17, 2010 at 3:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>> Hi all,
>>>
>>> Chukwa agent code is not intuitive to understand.  This is mainly the
>>> interface in Chukwa agent is over complicating the implementation.  I
>>> don't see a rationale for having each class to be an interface.
>>> Connector and Chukwa Sender are two interfaces which are not very
>>> useful to be interface.  It creates over complicated subsystem to
>>> maintain collector list in agent, connector and sender.  Ideally,
>>> there should be a single place for configuration source of truth.  I
>>> am leaning toward making those interface abstract classes.  Connector
>>> should be rename to something more meaningful like multiplexer or MUX
>>> for short.  AgentControlSocketListener could be refactor into a jersey
>>> like rest api, for easier to maintain the code base, and remove the
>>> 9093 protocol.
>>> This sounds like a major task.  I like to gather feedback to see if
>>> this change is necessary for creating more traction on chukwa
>>> development.
>>>
>>> regards,
>>> Eric
>>>
>>
>>
>>
>> --
>> Ari Rabkin [EMAIL PROTECTED]
>> UC Berkeley Computer Science Department
>>
>