Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> [VOTE] Direction for Hadoop development


Copy link to this message
-
Re: [VOTE] Direction for Hadoop development
On 12/07/2010 10:26 AM, Konstantin Shvachko wrote:
>> I no longer think we should add any new serialization implementations to
> the kernel.
>
> Not clear. Do you propose to keep current serialization(s) and not add new
> ones?
> Or do you propose to replace current serialization by abstract interfaces
> and move implementations to libraries?

We can't move existing serialization implementations to an optional
library without breaking compatibility.  Long-term that might be nice,
but I am not proposing that short-term.  Short-term I propose we avoid
adding new serialization implementations to the default classpath,
especially those that add new dependencies to every task.  Long-term we
might split library code into perhaps a few categories:
  - mandatory: this might include, e.g., IdentityMapper and
IdentityReducer, the default implementations.
  - back-compatible: the collection of library components that were
provided on the default classpath and can be enabled for back-compatible
behavior.
  - optional: components that jobs can optionally depend on.  This is
where new components that are not mandatory would be added.

Doug
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB