Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # dev >> Re: Incubator PMC/Board report for Jul 2013 ([ppmc])

Copy link to this message
Re: Incubator PMC/Board report for Jul 2013 ([ppmc])
I think it might be better to change it to what you had before.  :)

Chukwa code needs to be maintained to keep up with new versions of Hadoop.

On Jul 8, 2013, at 9:00 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> When using Chukwa as a generic log collection framework.  There is no
> change necessary.  When using Chukwa to monitor Hadoop cluster, then the
> data source to collect the data is different between Hadoop 1.x and 2.x.
> In our report, this is meant to apply our changes to monitor Hadoop 2.x.
> I have done some experimentation with Hadoop 2.x with Chukwa trunk.  Some
> test cases failed because Jobtracker does not exist in 2.x, hence Hadoop
> data collection needs to be updated.  We probably should aim for making
> those changes compatible with Hadoop 2.x.
> regards,
> Eric
> On Mon, Jul 8, 2013 at 7:29 PM, Alan D. Cabrera <[EMAIL PROTECTED]>wrote:
>> Awesome!  I'll update the report!
>> Regards,
>> Alan
>> On Jul 8, 2013, at 6:33 PM, 唐 颖 <[EMAIL PROTECTED]> wrote:
>>> Hi Eric,
>>> My team is now migrating our m/r from hadoop 1.0 to 2. 0 .
>>> And the old m/r can run on hadoop 2.0 without changing any source code ,
>> just need to re-compile on hadoop 2.0.
>>> The yarn in not a necessary if you want to run mr from hadoop 1.0 on
>> hadoop 2.0.
>>> --
>>> Best regards,
>>> Ivy Tang
>>> On 2013-7-8, at 上午1:03, Eric Yang <[EMAIL PROTECTED]> wrote:
>>>> Hadoop has evolved in a non straight forward release branches.  Chukwa
>>>> currently works with Hadoop 1.x branch.  For moving forward to Hadoop
>> 2.x,
>>>> the mapreduce jobs need to be using the new Yarn framework to implement
>>>> some of the missing pieces that Hadoop broke the compatibility.  HBase
>> has
>>>> done something similar for Hadoop compatibility layer.  Chukwa may have
>> to
>>>> do the same thing to keep up to date.  I don't know how to quantify the
>> man
>>>> hours needed to keep up to date.  It may be one or two weeks to work on
>> it
>>>> full time.  It may spread across several months, when we work on this
>> part
>>>> time.
>>>> The instrumentation points in MR v1 will needs to be migrated and
>>>> instrument Yarn.  It is going to be interesting work to monitor and
>> analyze
>>>> Yarn.  Therefore, the work may be interesting to people that would like
>> to
>>>> visualize yarn performance.  We probably should release 0.6 as bugfix
>>>> release before working on this.
>>>> regards,
>>>> Eric
>>>> On Sun, Jul 7, 2013 at 9:27 AM, Alan D. Cabrera <[EMAIL PROTECTED]
>>> wrote:
>>>>> On Jul 6, 2013, at 7:52 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>>>>> ChukwaChukwa is a log collection and analysis framework for Apache
>>>>> Hadoop clusters.Chukwa has been incubating since 2010-07-14.
>>>>>> Three most important issues to address in the move towards graduation:
>>>>>> 1. Keep code up-to-date with Hadoop trend
>>>>>> 2. Make another release
>>>>>> 3. Keep wiki page up to date and better documentation
>>>>> IMO, the number one item should be increasing an active community.
>>>>>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>>>>> aware of?
>>>>>> Chukwa code needs to be maintained to keep up with new versions of
>>>>> Hadoop.
>>>>> Can you characterize what is the scope of the work that needs to be
>> done?
>>>>> Especially in terms of estimated man hours needed to bring Chukwa up to
>>>>> date.
>>>>>> How has the community developed since the last report?
>>>>>> Committers are communicate more actively than previous quarter.
>>>>>> How has the project developed since the last report?
>>>>>> More contributors have open JIRA to request features and experiment
>> with
>>>>> code base.  Website is now svnpubsub enabled.  Javadocs are
>> regenerated to
>>>>> remove frame injection vulnerability.
>>>>>> Date of last release: January 2012