Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - Rejuvenate Hadoop 0.22 effort


Copy link to this message
-
Re: Rejuvenate Hadoop 0.22 effort
Konstantin Shvachko 2011-09-26, 01:02
Good news Roman!

About connecting with other projects.
HBase is compiling with 0.22.
For Pig there is
https://issues.apache.org/jira/browse/PIG-2277
For Hive created
https://issues.apache.org/jira/browse/HIVE-2468

The direction with Hive and Pig is to create shim layers for different versions.

Don't know about the API delta between .22 and .23 yet. I assume it is
less than 0.20 vs 0.22. But I may be wrong.

--Konstantin

On Fri, Sep 23, 2011 at 5:34 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> On Fri, Sep 16, 2011 at 2:21 AM, Konstantin Shvachko
> <[EMAIL PROTECTED]> wrote:
>> == TESTING =>> 5. I think Steve's idea of integrating 0.22 with Apache BigTop is
>> great. Will be glad to see any steps in this direction.
>
> The basic integration is done. We can produce fully functional RPM
> and DEB packages for Hadoop 0.22 release.
>
> This is good news. The bad news is that very few downstream components
> can be compiled against .22. And I'm not talking changes to versions, pom.xml
> and build.xml files. I'm talking API incompatibilities. Pig, Hive, HBase, Mahout
> all need to be modified to support .22. Before that's done -- there's
> little that
> can be done as far as stack validation is concerned.
>
> Given that work needs to be done in downstream components, I've got 2 questions:
>   1. do we know if the API delta between .22 and .23 is as
> significant as betwen
>   .22 and .20.2?
>
>   2. what's the common approach downstream to support multiple versions of
>   Hadoop APIs? Or is this even something that can be asked of all the
> components?
>
> Thanks,
> Roman.
>