Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # dev >> Simplify chukwa agent


Copy link to this message
-
Re: Simplify chukwa agent
+1 on refactoring the agent code and it's related interfaces at some
point. Implementing an adaptor seems a bit more confusing than it
ought to be. Some of the methods on Adaptor, like getCurrentStatus for
instance, do quite a bit more than their javadoc implies, or the
method name would lead you to believe.
On Sun, Oct 17, 2010 at 11:14 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> Hi Ari,
>
> The list of TODO from my side for Chukwa 0.5 is:
>
> - Update demux parsers (New parser for parsing hadoop metrics, job
> history 0.20+ log, and hadoop logs)
> - Update data analytics scripts using pig 0.8
> - Update documentation
>
> I am waiting on pig to release 0.8 which contains support for using
> HBase as storage.  This will enable chukwa data analytics on top of
> pig+HBase.
> After pig 0.8 is released, then Chukwa 0.5 release will be ready.  I
> agree that refactoring the interface in Chukwa Agent could wait until
> Chukwa 0.5 is released.
>
> regards,
> Eric
>
> On Sun, Oct 17, 2010 at 10:54 PM, Ariel Rabkin <[EMAIL PROTECTED]> wrote:
>> Howdy.
>>
>> I agree that that part of the code is pretty snarly. Not sure how
>> urgent a priority it is to fix.
>>
>> I think a better short-term goal might be doing a 0.5 release. How
>> close are we to that?
>>
>> --Ari
>>
>> On Sun, Oct 17, 2010 at 3:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>> Hi all,
>>>
>>> Chukwa agent code is not intuitive to understand.  This is mainly the
>>> interface in Chukwa agent is over complicating the implementation.  I
>>> don't see a rationale for having each class to be an interface.
>>> Connector and Chukwa Sender are two interfaces which are not very
>>> useful to be interface.  It creates over complicated subsystem to
>>> maintain collector list in agent, connector and sender.  Ideally,
>>> there should be a single place for configuration source of truth.  I
>>> am leaning toward making those interface abstract classes.  Connector
>>> should be rename to something more meaningful like multiplexer or MUX
>>> for short.  AgentControlSocketListener could be refactor into a jersey
>>> like rest api, for easier to maintain the code base, and remove the
>>> 9093 protocol.
>>> This sounds like a major task.  I like to gather feedback to see if
>>> this change is necessary for creating more traction on chukwa
>>> development.
>>>
>>> regards,
>>> Eric
>>>
>>
>>
>>
>> --
>> Ari Rabkin [EMAIL PROTECTED]
>> UC Berkeley Computer Science Department
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB