Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # dev >> Re: Incubator PMC/Board report for Jul 2013 ([ppmc])


+
Eric Yang 2013-07-03, 03:18
+
Eric Yang 2013-07-07, 02:52
+
Alan D. Cabrera 2013-07-07, 16:27
+
Eric Yang 2013-07-07, 17:03
+
Alan D. Cabrera 2013-07-07, 18:39
+
Alan Cabrera 2013-07-08, 15:25
+
唐 颖 2013-07-09, 01:33
+
Alan D. Cabrera 2013-07-09, 02:29
+
Eric Yang 2013-07-09, 04:00
Copy link to this message
-
Re: Incubator PMC/Board report for Jul 2013 ([ppmc])
I think it might be better to change it to what you had before.  :)

Chukwa code needs to be maintained to keep up with new versions of Hadoop.
Regards,
Alan

On Jul 8, 2013, at 9:00 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> When using Chukwa as a generic log collection framework.  There is no
> change necessary.  When using Chukwa to monitor Hadoop cluster, then the
> data source to collect the data is different between Hadoop 1.x and 2.x.
> In our report, this is meant to apply our changes to monitor Hadoop 2.x.
> I have done some experimentation with Hadoop 2.x with Chukwa trunk.  Some
> test cases failed because Jobtracker does not exist in 2.x, hence Hadoop
> data collection needs to be updated.  We probably should aim for making
> those changes compatible with Hadoop 2.x.
>
> regards,
> Eric
>
>
> On Mon, Jul 8, 2013 at 7:29 PM, Alan D. Cabrera <[EMAIL PROTECTED]>wrote:
>
>> Awesome!  I'll update the report!
>>
>>
>> Regards,
>> Alan
>>
>> On Jul 8, 2013, at 6:33 PM, 唐 颖 <[EMAIL PROTECTED]> wrote:
>>
>>> Hi Eric,
>>>
>>> My team is now migrating our m/r from hadoop 1.0 to 2. 0 .
>>> And the old m/r can run on hadoop 2.0 without changing any source code ,
>> just need to re-compile on hadoop 2.0.
>>> The yarn in not a necessary if you want to run mr from hadoop 1.0 on
>> hadoop 2.0.
>>> --
>>> Best regards,
>>>
>>> Ivy Tang
>>>
>>> On 2013-7-8, at 上午1:03, Eric Yang <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hadoop has evolved in a non straight forward release branches.  Chukwa
>>>> currently works with Hadoop 1.x branch.  For moving forward to Hadoop
>> 2.x,
>>>> the mapreduce jobs need to be using the new Yarn framework to implement
>>>> some of the missing pieces that Hadoop broke the compatibility.  HBase
>> has
>>>> done something similar for Hadoop compatibility layer.  Chukwa may have
>> to
>>>> do the same thing to keep up to date.  I don't know how to quantify the
>> man
>>>> hours needed to keep up to date.  It may be one or two weeks to work on
>> it
>>>> full time.  It may spread across several months, when we work on this
>> part
>>>> time.
>>>>
>>>> The instrumentation points in MR v1 will needs to be migrated and
>>>> instrument Yarn.  It is going to be interesting work to monitor and
>> analyze
>>>> Yarn.  Therefore, the work may be interesting to people that would like
>> to
>>>> visualize yarn performance.  We probably should release 0.6 as bugfix
>>>> release before working on this.
>>>>
>>>> regards,
>>>> Eric
>>>>
>>>>
>>>> On Sun, Jul 7, 2013 at 9:27 AM, Alan D. Cabrera <[EMAIL PROTECTED]
>>> wrote:
>>>>
>>>>>
>>>>> On Jul 6, 2013, at 7:52 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> ChukwaChukwa is a log collection and analysis framework for Apache
>>>>> Hadoop clusters.Chukwa has been incubating since 2010-07-14.
>>>>>>
>>>>>> Three most important issues to address in the move towards graduation:
>>>>>>
>>>>>> 1. Keep code up-to-date with Hadoop trend
>>>>>> 2. Make another release
>>>>>> 3. Keep wiki page up to date and better documentation
>>>>>
>>>>> IMO, the number one item should be increasing an active community.
>>>>>
>>>>>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>>>>> aware of?
>>>>>>
>>>>>> Chukwa code needs to be maintained to keep up with new versions of
>>>>> Hadoop.
>>>>>
>>>>> Can you characterize what is the scope of the work that needs to be
>> done?
>>>>> Especially in terms of estimated man hours needed to bring Chukwa up to
>>>>> date.
>>>>>
>>>>>> How has the community developed since the last report?
>>>>>>
>>>>>> Committers are communicate more actively than previous quarter.
>>>>>>
>>>>>> How has the project developed since the last report?
>>>>>>
>>>>>> More contributors have open JIRA to request features and experiment
>> with
>>>>> code base.  Website is now svnpubsub enabled.  Javadocs are
>> regenerated to
>>>>> remove frame injection vulnerability.
>>>>>>
>>>>>> Date of last release: January 2012
+
Marvin 2013-07-07, 11:12
+
Marvin 2013-10-06, 16:05
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB