Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> Status on MAPREDUCE-279 and plans forward


+
Arun C Murthy 2011-06-10, 19:44
Copy link to this message
-
Re: Status on MAPREDUCE-279 and plans forward
Hi Arun,

Great to hear that MR2 is nearly ready for trunk.

MR2 is being built using Maven, so step 5 is about getting Maven to
build all of MR I presume? The svn unsplit (HADOOP-7106) and
Mavenization work in HADOOP-6671 should make this approach more
tractable. Or were you thinking that there would be a hybrid Ant/Maven
build during a transition period?

Also, for step 3, what did you think of the directory layout I
suggested in http://bit.ly/mr-279-maven-layout?

Cheers,
Tom

On Fri, Jun 10, 2011 at 12:44 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> Another heads up.
>
> Things are going well, we have done scale testing (350 nodes) and continue
> to do more.
>
> We think we are nearly ready to merge MR-279 into trunk (yaay!).
>
> A proposal for plan forward:
>
> 1. I'd like to get MAPREDUCE-2400 committed to both trunk and MR-279. This
> is a very important patch by Tom which allows to submit jobs to both JT and
> the new RM - thanks Tom!
> 2. Check-in the 'yarn' subfolder which is just the ResourceManager and
> NodeManager i.e. compute fabric to trunk.
> 3. Rework the src tree in trunk to look like the MR-279 branch to ensure no
> changes to trunk are lost - this will ensure separation between the
> 'classic' framework (JT/TT) and the MR runtime (map/reduce/sort/shuffle
> etc.)
> 4. Re-play the 4-5 patches needed to the MR runtime from the MR-279 branch.
> 5. Change build scripts to reflect the new reality.
>
> In terms of timelines, I'm hoping we can get this done in the coming 2-3
> weeks.
>
> Thoughts?
>
> thanks,
> Arun
>
+
Arun C Murthy 2011-06-11, 08:10
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB