Hi All,

I am planning to commit HBASE-5955 (https://issues.apache.org/jira/browse/HBASE-5955) to 0.94.1.

This upgrades Guava to 11.0.2, which will likely cause problems with version of Hadoop >= 0.22 and < 2.0.0.
(Hadoop 0.22 introduced Guava at version 9 and 2.0.0 updates the version to 11.0.2)

Please comment on the issue, if there are no objections I'll commit this soon.
Thanks.
-- Lars

On Sat, Jun 30, 2012 at 6:22 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> Hi All,
>
> I am planning to commit HBASE-5955 (https://issues.apache.org/jira/browse/HBASE-5955) to 0.94.1.
>
> This upgrades Guava to 11.0.2, which will likely cause problems with version of Hadoop >= 0.22 and < 2.0.0.
> (Hadoop 0.22 introduced Guava at version 9 and 2.0.0 updates the version to 11.0.2)
>
> Please comment on the issue, if there are no objections I'll commit this soon.
>

That seems fine.

We support 1.0 and 2.0.  Do we work against 0.22 w/o modification as is?

(What about CDH?  The Hortonworks distro should be fine).

St.Ack

I think CDH is either based on Hadoop 1 in CDH3 (which did not have Guava, so it cannot conflict) or Hadoop 2in CDH4 (which is Guava version 11.0.2).
I'll let the Cloudera folks make a final statement. I do not know about Hortonworks.
-- Lars

----- Original Message -----
From: Stack <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
Cc:
Sent: Saturday, June 30, 2012 9:39 AM
Subject: Re: HBASE-5955

On Sat, Jun 30, 2012 at 6:22 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> Hi All,
>
> I am planning to commit HBASE-5955 (https://issues.apache.org/jira/browse/HBASE-5955) to 0.94.1.
>
> This upgrades Guava to 11.0.2, which will likely cause problems with version of Hadoop >= 0.22 and < 2.0.0.
> (Hadoop 0.22 introduced Guava at version 9 and 2.0.0 updates the version to 11.0.2)
>
> Please comment on the issue, if there are no objections I'll commit this soon.
>

That seems fine.

We support 1.0 and 2.0.  Do we work against 0.22 w/o modification as is?

(What about CDH?  The Hortonworks distro should be fine).

St.Ack

CDH3 does use Guava. I don't know the version, but can see com.google imports in various backported HDFS patches.

    - Andy

On Jun 30, 2012, at 9:45 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> I think CDH is either based on Hadoop 1 in CDH3 (which did not have Guava, so it cannot conflict) or Hadoop 2in CDH4 (which is Guava version 11.0.2).
> I'll let the Cloudera folks make a final statement. I do not know about Hortonworks.
>
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Stack <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
> Cc:
> Sent: Saturday, June 30, 2012 9:39 AM
> Subject: Re: HBASE-5955
>
> On Sat, Jun 30, 2012 at 6:22 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>> Hi All,
>>
>> I am planning to commit HBASE-5955 (https://issues.apache.org/jira/browse/HBASE-5955) to 0.94.1.
>>
>> This upgrades Guava to 11.0.2, which will likely cause problems with version of Hadoop >= 0.22 and < 2.0.0.
>> (Hadoop 0.22 introduced Guava at version 9 and 2.0.0 updates the version to 11.0.2)
>>
>> Please comment on the issue, if there are no objections I'll commit this soon.
>>
>
> That seems fine.
>
> We support 1.0 and 2.0.  Do we work against 0.22 w/o modification as is?
>
> (What about CDH?  The Hortonworks distro should be fine).
>
> St.Ack
>

Each CDH tries to harmonize (all use the same version) of libraries used:

CDH3's hbase has guava-r06.jar, and guava-r09-jarjar.jar
CDH4.0.0's hbase has guava-11.0.2.jar

Jon.

On Sat, Jun 30, 2012 at 10:11 AM, Andrew Purtell
<[EMAIL PROTECTED]>wrote:

> CDH3 does use Guava. I don't know the version, but can see com.google
> imports in various backported HDFS patches.
>
>    - Andy
>
> On Jun 30, 2012, at 9:45 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
> > I think CDH is either based on Hadoop 1 in CDH3 (which did not have
> Guava, so it cannot conflict) or Hadoop 2in CDH4 (which is Guava version
> 11.0.2).
> > I'll let the Cloudera folks make a final statement. I do not know about
> Hortonworks.
> >
> >
> > -- Lars
> >
> >
> >
> > ----- Original Message -----
> > From: Stack <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
> > Cc:
> > Sent: Saturday, June 30, 2012 9:39 AM
> > Subject: Re: HBASE-5955
> >
> > On Sat, Jun 30, 2012 at 6:22 PM, lars hofhansl <[EMAIL PROTECTED]>
> wrote:
> >> Hi All,
> >>
> >> I am planning to commit HBASE-5955 (
> https://issues.apache.org/jira/browse/HBASE-5955) to 0.94.1.
> >>
> >> This upgrades Guava to 11.0.2, which will likely cause problems with
> version of Hadoop >= 0.22 and < 2.0.0.
> >> (Hadoop 0.22 introduced Guava at version 9 and 2.0.0 updates the
> version to 11.0.2)
> >>
> >> Please comment on the issue, if there are no objections I'll commit
> this soon.
> >>
> >
> > That seems fine.
> >
> > We support 1.0 and 2.0.  Do we work against 0.22 w/o modification as is?
> >
> > (What about CDH?  The Hortonworks distro should be fine).
> >
> > St.Ack
> >
>

--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]

So will Guava 11.0.2 in hbase 0.94 be a problem? Should we not update Guava (at least not until 0.96)?

-- Lars
----- Original Message -----
From: Jonathan Hsieh <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc:
Sent: Saturday, June 30, 2012 1:47 PM
Subject: Re: HBASE-5955

Each CDH tries to harmonize (all use the same version) of libraries used:

CDH3's hbase has guava-r06.jar, and guava-r09-jarjar.jar
CDH4.0.0's hbase has guava-11.0.2.jar

Jon.

On Sat, Jun 30, 2012 at 10:11 AM, Andrew Purtell
<[EMAIL PROTECTED]>wrote:

> CDH3 does use Guava. I don't know the version, but can see com.google
> imports in various backported HDFS patches.
>
>    - Andy
>
> On Jun 30, 2012, at 9:45 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
> > I think CDH is either based on Hadoop 1 in CDH3 (which did not have
> Guava, so it cannot conflict) or Hadoop 2in CDH4 (which is Guava version
> 11.0.2).
> > I'll let the Cloudera folks make a final statement. I do not know about
> Hortonworks.
> >
> >
> > -- Lars
> >
> >
> >
> > ----- Original Message -----
> > From: Stack <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
> > Cc:
> > Sent: Saturday, June 30, 2012 9:39 AM
> > Subject: Re: HBASE-5955
> >
> > On Sat, Jun 30, 2012 at 6:22 PM, lars hofhansl <[EMAIL PROTECTED]>
> wrote:
> >> Hi All,
> >>
> >> I am planning to commit HBASE-5955 (
> https://issues.apache.org/jira/browse/HBASE-5955) to 0.94.1.
> >>
> >> This upgrades Guava to 11.0.2, which will likely cause problems with
> version of Hadoop >= 0.22 and < 2.0.0.
> >> (Hadoop 0.22 introduced Guava at version 9 and 2.0.0 updates the
> version to 11.0.2)
> >>
> >> Please comment on the issue, if there are no objections I'll commit
> this soon.
> >>
> >
> > That seems fine.
> >
> > We support 1.0 and 2.0.  Do we work against 0.22 w/o modification as is?
> >
> > (What about CDH?  The Hortonworks distro should be fine).
> >
> > St.Ack
> >
>

--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]

On Sat, Jun 30, 2012 at 5:24 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> So will Guava 11.0.2 in hbase 0.94 be a problem? Should we not update Guava (at least not until 0.96)?

In general, I don't think there's good answer to how dependencies
are supposed to be managed wrt. version numbers. After all, all
of the Hadoop ecosystem projects have to deal with the fact that
there's at least half a dozen different branches of Hadoop they have
to work with. It is virtually guaranteed that whatever you do you will
end up incompatible with one of those builds and most likely the
version of a dependency that wins will be the one that Hadoop is using.

IOW, whatever is specified in the POM file of individual projects is
likely to be overridden anyway and the best the could be done
is to avoid known gaping incompatibility holes (of which I'm not
aware of too many).

Thanks,
Roman.

> IOW, whatever is specified in the POM file of individual projects is
> likely to be overridden anyway and the best the could be done
> is to avoid known gaping incompatibility holes (of which I'm not
> aware of too many).

Fair enough, but Guava is a particular problem. Map eviction listeners
completely change somewhere between r09 and r11. It could
theoretically be possible to handle that with reflection but that
would be a lot of reflection, in performance sensitive code paths, and
likely to fail only at runtime/in production if there is a mistake or
another change. So all of Hadoop should get on the same page with
this. Sadly I see Giraph just moved to 12 while everyone else is on
11.

On Sat, Jun 30, 2012 at 7:17 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> On Sat, Jun 30, 2012 at 5:24 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>> So will Guava 11.0.2 in hbase 0.94 be a problem? Should we not update Guava (at least not until 0.96)?
>
> In general, I don't think there's good answer to how dependencies
> are supposed to be managed wrt. version numbers. After all, all
> of the Hadoop ecosystem projects have to deal with the fact that
> there's at least half a dozen different branches of Hadoop they have
> to work with. It is virtually guaranteed that whatever you do you will
> end up incompatible with one of those builds and most likely the
> version of a dependency that wins will be the one that Hadoop is using.
>
> IOW, whatever is specified in the POM file of individual projects is
> likely to be overridden anyway and the best the could be done
> is to avoid known gaping incompatibility holes (of which I'm not
> aware of too many).
>
> Thanks,
> Roman.

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet
Hein (via Tom White)

That is the best argument I heard so far. I'll commit the patch today.

Thanks.

-- Lars

Roman Shaposhnik <[EMAIL PROTECTED]> schrieb:

>On Sat, Jun 30, 2012 at 5:24 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>> So will Guava 11.0.2 in hbase 0.94 be a problem? Should we not update Guava (at least not until 0.96)?
>
>In general, I don't think there's good answer to how dependencies
>are supposed to be managed wrt. version numbers. After all, all
>of the Hadoop ecosystem projects have to deal with the fact that
>there's at least half a dozen different branches of Hadoop they have
>to work with. It is virtually guaranteed that whatever you do you will
>end up incompatible with one of those builds and most likely the
>version of a dependency that wins will be the one that Hadoop is using.
>
>IOW, whatever is specified in the POM file of individual projects is
>likely to be overridden anyway and the best the could be done
>is to avoid known gaping incompatibility holes (of which I'm not
>aware of too many).
>
>Thanks,
>Roman.

IIRC, we also shaded Guava in CDH3 into something like
org.apache.hadoop.thirdparty.com.google.guava.

-Todd

On Mon, Jul 2, 2012 at 9:28 AM, Lars H <[EMAIL PROTECTED]> wrote:

> That is the best argument I heard so far. I'll commit the patch today.
>
> Thanks.
>
> -- Lars
>
> Roman Shaposhnik <[EMAIL PROTECTED]> schrieb:
>
> >On Sat, Jun 30, 2012 at 5:24 PM, lars hofhansl <[EMAIL PROTECTED]>
> wrote:
> >> So will Guava 11.0.2 in hbase 0.94 be a problem? Should we not update
> Guava (at least not until 0.96)?
> >
> >In general, I don't think there's good answer to how dependencies
> >are supposed to be managed wrt. version numbers. After all, all
> >of the Hadoop ecosystem projects have to deal with the fact that
> >there's at least half a dozen different branches of Hadoop they have
> >to work with. It is virtually guaranteed that whatever you do you will
> >end up incompatible with one of those builds and most likely the
> >version of a dependency that wins will be the one that Hadoop is using.
> >
> >IOW, whatever is specified in the POM file of individual projects is
> >likely to be overridden anyway and the best the could be done
> >is to avoid known gaping incompatibility holes (of which I'm not
> >aware of too many).
> >
> >Thanks,
> >Roman.
>

--
Todd Lipcon
Software Engineer, Cloudera