Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> [DISCUSSION] Combine committer lists in Common/HDFS/MapReduce


Copy link to this message
-
Re: [DISCUSSION] Combine committer lists in Common/HDFS/MapReduce


On 8/9/2010 9:26 AM, Doug Cutting wrote:
 > On 08/08/2010 12:21 PM, Arun C Murthy wrote:
 >> This of course begs a larger question - should we just merge Common,
 >> HDFS&  Map-Reduce together and be done with?
 >
 > I think there's still a reasonable long-term goal to split MapReduce
 > from HDFS, so that they can release separately and are maintained by
 > separate teams.  So I believe a strong division of these code trees and
 > release artifacts should remain.

I think that eventually when we solve the backward compatibility issue,
with Avro, we can make MR and HDFS different products.
Even though right now they are not, it would be a step back to merge
them back.

As we look forward to making two different products, shouldn't there be
two separate sets of committers?
Of course, as long as they are not really separated it makes sense to
have single committers pool.

 > I'd like to get rid of Common.  It could either be merged into HDFS or
 > gradually whittled away to nothing.  I'd prefer the latter.  If we move
 > to different RPC and serialization systems (e.g., Avro) then Common's
 > io, and ipc packages might be removed.  Configuration might be
 > replaced/merged with Jakarta Commons Configuration
 > (http://commons.apache.org/configuration/).  Similarly, the metrics and
 > fs packages might be moved to Jakarta Commons.  Such changes might be
 > hard to do back-compatibly, however.

Great idea. We should do this.

 > I don't see that merging the Jira databases or mailing lists for HDFS
 > and MapReduce offers big advantages.  The redundant, coordinated Jira's
 > tend to be between Common the others, no?

Yes.
--Konstantin