Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - [VOTE] Direction for Hadoop development


Copy link to this message
-
Re: [VOTE] Direction for Hadoop development
Doug Cutting 2010-12-08, 18:55
On 12/07/2010 10:26 AM, Konstantin Shvachko wrote:
>> I no longer think we should add any new serialization implementations to
> the kernel.
>
> Not clear. Do you propose to keep current serialization(s) and not add new
> ones?
> Or do you propose to replace current serialization by abstract interfaces
> and move implementations to libraries?

We can't move existing serialization implementations to an optional
library without breaking compatibility.  Long-term that might be nice,
but I am not proposing that short-term.  Short-term I propose we avoid
adding new serialization implementations to the default classpath,
especially those that add new dependencies to every task.  Long-term we
might split library code into perhaps a few categories:
  - mandatory: this might include, e.g., IdentityMapper and
IdentityReducer, the default implementations.
  - back-compatible: the collection of library components that were
provided on the default classpath and can be enabled for back-compatible
behavior.
  - optional: components that jobs can optionally depend on.  This is
where new components that are not mandatory would be added.

Doug