Unless there is objection, in the next day or so, I'd like to revert
the hadoop that is in hbase trunk and replace it with hadoop 0.20.2.
Currently hbase trunk bundles the head of the 0.21 hadoop branch and
will only run atop an hadoop built out of this hadoop 0.21 branch.

Since the plan is that the next release of hbase will run on hadoop
0.20.x and the major hadoop release that follows hadoop 0.20 -- 0.21
or 0.22, its version is still TBD -- AND given that the next major
hadoop release would seem to be a few months out, having our trunk
bundle hadoop 0.21 unnecessarily constrains where current hbase trunk
can run.

Even though the bundled hadoop 0.21 has the critical hdfs-265 working
flush/sync feature, its looking like hard work by the fellas who are
on the hdfs-200, hdfs-862 and hdfs-142 patchset will likely get us
data durability on a (patched) hadoop 0.20.x some time soon.  Having
hbase trunk so it can work on the patched hadoop 0.20.x will help the
testing of the data durability effort.

Jean-Daniel, Ryan, and I have already had a pow-wow about this and
think this is the way to go for now (The revert notion was originally
Ryans -- See HBASE-2255).

If you've been running hbase TRUNK and an hadoop 0.21, unfortunately,
this means that you'll need to copy your data out of your cluster, do
the downgrade, and then copy your data back in.  If you need help with
this effort, give us a shout and we'll help you out.

Thanks,
St.Ack

+1 from me.

To help out the guys who are already running on unreleased Hadoop, would it
be possible to build a simple shim layer that would allow it to work with
either? We did this in Hive and it seems to be working well (few complaints
so far)

-Todd

On Thu, Mar 18, 2010 at 11:28 AM, Stack <[EMAIL PROTECTED]> wrote:

> Unless there is objection, in the next day or so, I'd like to revert
> the hadoop that is in hbase trunk and replace it with hadoop 0.20.2.
> Currently hbase trunk bundles the head of the 0.21 hadoop branch and
> will only run atop an hadoop built out of this hadoop 0.21 branch.
>
> Since the plan is that the next release of hbase will run on hadoop
> 0.20.x and the major hadoop release that follows hadoop 0.20 -- 0.21
> or 0.22, its version is still TBD -- AND given that the next major
> hadoop release would seem to be a few months out, having our trunk
> bundle hadoop 0.21 unnecessarily constrains where current hbase trunk
> can run.
>
> Even though the bundled hadoop 0.21 has the critical hdfs-265 working
> flush/sync feature, its looking like hard work by the fellas who are
> on the hdfs-200, hdfs-862 and hdfs-142 patchset will likely get us
> data durability on a (patched) hadoop 0.20.x some time soon.  Having
> hbase trunk so it can work on the patched hadoop 0.20.x will help the
> testing of the data durability effort.
>
> Jean-Daniel, Ryan, and I have already had a pow-wow about this and
> think this is the way to go for now (The revert notion was originally
> Ryans -- See HBASE-2255).
>
> If you've been running hbase TRUNK and an hadoop 0.21, unfortunately,
> this means that you'll need to copy your data out of your cluster, do
> the downgrade, and then copy your data back in.  If you need help with
> this effort, give us a shout and we'll help you out.
>
> Thanks,
> St.Ack
>

--
Todd Lipcon
Software Engineer, Cloudera

I'm obviously +1

@Todd, that's the plan https://issues.apache.org/jira/browse/HBASE-2233

J-D

On Thu, Mar 18, 2010 at 11:32 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> +1 from me.
>
> To help out the guys who are already running on unreleased Hadoop, would it
> be possible to build a simple shim layer that would allow it to work with
> either? We did this in Hive and it seems to be working well (few complaints
> so far)
>
> -Todd
>
> On Thu, Mar 18, 2010 at 11:28 AM, Stack <[EMAIL PROTECTED]> wrote:
>
>> Unless there is objection, in the next day or so, I'd like to revert
>> the hadoop that is in hbase trunk and replace it with hadoop 0.20.2.
>> Currently hbase trunk bundles the head of the 0.21 hadoop branch and
>> will only run atop an hadoop built out of this hadoop 0.21 branch.
>>
>> Since the plan is that the next release of hbase will run on hadoop
>> 0.20.x and the major hadoop release that follows hadoop 0.20 -- 0.21
>> or 0.22, its version is still TBD -- AND given that the next major
>> hadoop release would seem to be a few months out, having our trunk
>> bundle hadoop 0.21 unnecessarily constrains where current hbase trunk
>> can run.
>>
>> Even though the bundled hadoop 0.21 has the critical hdfs-265 working
>> flush/sync feature, its looking like hard work by the fellas who are
>> on the hdfs-200, hdfs-862 and hdfs-142 patchset will likely get us
>> data durability on a (patched) hadoop 0.20.x some time soon.  Having
>> hbase trunk so it can work on the patched hadoop 0.20.x will help the
>> testing of the data durability effort.
>>
>> Jean-Daniel, Ryan, and I have already had a pow-wow about this and
>> think this is the way to go for now (The revert notion was originally
>> Ryans -- See HBASE-2255).
>>
>> If you've been running hbase TRUNK and an hadoop 0.21, unfortunately,
>> this means that you'll need to copy your data out of your cluster, do
>> the downgrade, and then copy your data back in.  If you need help with
>> this effort, give us a shout and we'll help you out.
>>
>> Thanks,
>> St.Ack
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

+1

Thanks, this means I can get off of branch for dev and move back to trunk.

  - Andy

> From: Stack <[EMAIL PROTECTED]>
> Subject: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to hadoop  0.20.x (from head of the hadoop 0.21 branch)
> To: "HBase Dev List" <[EMAIL PROTECTED]>
> Date: Thursday, March 18, 2010, 11:28 AM
>
> Unless there is objection, in the next day or so, I'd like
> to revert the hadoop that is in hbase trunk and replace it
> with hadoop 0.20.2.

+1

It's the best option in order to be able to release hbase-0.21.
 
We're already running on hdfs-0.21 and mapreduce-0.21 so if there won't be any real blocking issues we're going to stick with 0.21.

HBASE-2233 is appealing so we're for it unless it would take too much work, which doesn't seem to be the case.

Cosmin

On Mar 18, 2010, at 8:57 PM, Andrew Purtell wrote:

> +1
>
> Thanks, this means I can get off of branch for dev and move back to trunk.
>
>  - Andy
>
>> From: Stack <[EMAIL PROTECTED]>
>> Subject: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to hadoop  0.20.x (from head of the hadoop 0.21 branch)
>> To: "HBase Dev List" <[EMAIL PROTECTED]>
>> Date: Thursday, March 18, 2010, 11:28 AM
>>
>> Unless there is objection, in the next day or so, I'd like
>> to revert the hadoop that is in hbase trunk and replace it
>> with hadoop 0.20.2.
>
>
>
>
>
>

+1 from me and from FB.

> -----Original Message-----
> From: Cosmin Lehene [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, March 18, 2010 1:14 PM
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Subject: Re: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to
> hadoop 0.20.x (from head of the hadoop 0.21 branch)
>
> +1
>
> It's the best option in order to be able to release hbase-0.21.
>
> We're already running on hdfs-0.21 and mapreduce-0.21 so if there won't
> be any real blocking issues we're going to stick with 0.21.
>
> HBASE-2233 is appealing so we're for it unless it would take too much
> work, which doesn't seem to be the case.
>
> Cosmin
>
> On Mar 18, 2010, at 8:57 PM, Andrew Purtell wrote:
>
> > +1
> >
> > Thanks, this means I can get off of branch for dev and move back to
> trunk.
> >
> >  - Andy
> >
> >> From: Stack <[EMAIL PROTECTED]>
> >> Subject: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to
> hadoop  0.20.x (from head of the hadoop 0.21 branch)
> >> To: "HBase Dev List" <[EMAIL PROTECTED]>
> >> Date: Thursday, March 18, 2010, 11:28 AM
> >>
> >> Unless there is objection, in the next day or so, I'd like
> >> to revert the hadoop that is in hbase trunk and replace it
> >> with hadoop 0.20.2.
> >
> >
> >
> >
> >
> >

As one of the main drivers behind it, I'm majorly +1.  I think we can
get durability out of HDFS-200 (and a few other patches) and stay on a
fully released, tested, and (most importantly) trusted Hadoop release.
 Distributed filesystems are hard, and there is a reason there are
very few actually in real production use out there.  Staying on a
known good release is the way to go for now I think.

On Thu, Mar 18, 2010 at 2:10 PM, Jonathan Gray <[EMAIL PROTECTED]> wrote:
> +1 from me and from FB.
>
>> -----Original Message-----
>> From: Cosmin Lehene [mailto:[EMAIL PROTECTED]]
>> Sent: Thursday, March 18, 2010 1:14 PM
>> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
>> Subject: Re: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to
>> hadoop 0.20.x (from head of the hadoop 0.21 branch)
>>
>> +1
>>
>> It's the best option in order to be able to release hbase-0.21.
>>
>> We're already running on hdfs-0.21 and mapreduce-0.21 so if there won't
>> be any real blocking issues we're going to stick with 0.21.
>>
>> HBASE-2233 is appealing so we're for it unless it would take too much
>> work, which doesn't seem to be the case.
>>
>> Cosmin
>>
>> On Mar 18, 2010, at 8:57 PM, Andrew Purtell wrote:
>>
>> > +1
>> >
>> > Thanks, this means I can get off of branch for dev and move back to
>> trunk.
>> >
>> >  - Andy
>> >
>> >> From: Stack <[EMAIL PROTECTED]>
>> >> Subject: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to
>> hadoop  0.20.x (from head of the hadoop 0.21 branch)
>> >> To: "HBase Dev List" <[EMAIL PROTECTED]>
>> >> Date: Thursday, March 18, 2010, 11:28 AM
>> >>
>> >> Unless there is objection, in the next day or so, I'd like
>> >> to revert the hadoop that is in hbase trunk and replace it
>> >> with hadoop 0.20.2.
>> >
>> >
>> >
>> >
>> >
>> >
>
>

+1.

dhruba
On Thu, Mar 18, 2010 at 2:22 PM, Ryan Rawson <[EMAIL PROTECTED]> wrote:

> As one of the main drivers behind it, I'm majorly +1.  I think we can
> get durability out of HDFS-200 (and a few other patches) and stay on a
> fully released, tested, and (most importantly) trusted Hadoop release.
>  Distributed filesystems are hard, and there is a reason there are
> very few actually in real production use out there.  Staying on a
> known good release is the way to go for now I think.
>
> On Thu, Mar 18, 2010 at 2:10 PM, Jonathan Gray <[EMAIL PROTECTED]> wrote:
> > +1 from me and from FB.
> >
> >> -----Original Message-----
> >> From: Cosmin Lehene [mailto:[EMAIL PROTECTED]]
> >> Sent: Thursday, March 18, 2010 1:14 PM
> >> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> >> Subject: Re: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to
> >> hadoop 0.20.x (from head of the hadoop 0.21 branch)
> >>
> >> +1
> >>
> >> It's the best option in order to be able to release hbase-0.21.
> >>
> >> We're already running on hdfs-0.21 and mapreduce-0.21 so if there won't
> >> be any real blocking issues we're going to stick with 0.21.
> >>
> >> HBASE-2233 is appealing so we're for it unless it would take too much
> >> work, which doesn't seem to be the case.
> >>
> >> Cosmin
> >>
> >> On Mar 18, 2010, at 8:57 PM, Andrew Purtell wrote:
> >>
> >> > +1
> >> >
> >> > Thanks, this means I can get off of branch for dev and move back to
> >> trunk.
> >> >
> >> >  - Andy
> >> >
> >> >> From: Stack <[EMAIL PROTECTED]>
> >> >> Subject: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to
> >> hadoop  0.20.x (from head of the hadoop 0.21 branch)
> >> >> To: "HBase Dev List" <[EMAIL PROTECTED]>
> >> >> Date: Thursday, March 18, 2010, 11:28 AM
> >> >>
> >> >> Unless there is objection, in the next day or so, I'd like
> >> >> to revert the hadoop that is in hbase trunk and replace it
> >> >> with hadoop 0.20.2.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >
> >
>

--
Connect to me at http://www.facebook.com/dhruba

I've committed this as SVN revision # 926397.

As part of the change I built this version of hadoop:
http://github.com/ryanobjc/hadoop-common/commits/hadoop-0.20.2-with-200-826

And it is published:
http://people.apache.org/~rawson/repo/org/apache/hadoop/hadoop-core/0.20.2-with-200-826/
and
http://people.apache.org/~rawson/repo/org/apache/hadoop/hadoop-test/0.20.2-with-200-826/

So to build a hadoop with the precise version that HBase trunk now
depends on, you can git clone my repo above and do 'ant tar' to get
yourself a hadoop 0.20.2 + HDFS-200/826.

The tests should pass, but some of the sync-oriented tests were either
crippled (by closing the file) or disabled (TestHLog#testSync for
example).

Enjoy :-)
-ryan

On Thu, Mar 18, 2010 at 5:24 PM, Dhruba Borthakur <[EMAIL PROTECTED]> wrote:
> +1.
>
> dhruba
>
>
> On Thu, Mar 18, 2010 at 2:22 PM, Ryan Rawson <[EMAIL PROTECTED]> wrote:
>
>> As one of the main drivers behind it, I'm majorly +1.  I think we can
>> get durability out of HDFS-200 (and a few other patches) and stay on a
>> fully released, tested, and (most importantly) trusted Hadoop release.
>>  Distributed filesystems are hard, and there is a reason there are
>> very few actually in real production use out there.  Staying on a
>> known good release is the way to go for now I think.
>>
>> On Thu, Mar 18, 2010 at 2:10 PM, Jonathan Gray <[EMAIL PROTECTED]> wrote:
>> > +1 from me and from FB.
>> >
>> >> -----Original Message-----
>> >> From: Cosmin Lehene [mailto:[EMAIL PROTECTED]]
>> >> Sent: Thursday, March 18, 2010 1:14 PM
>> >> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
>> >> Subject: Re: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to
>> >> hadoop 0.20.x (from head of the hadoop 0.21 branch)
>> >>
>> >> +1
>> >>
>> >> It's the best option in order to be able to release hbase-0.21.
>> >>
>> >> We're already running on hdfs-0.21 and mapreduce-0.21 so if there won't
>> >> be any real blocking issues we're going to stick with 0.21.
>> >>
>> >> HBASE-2233 is appealing so we're for it unless it would take too much
>> >> work, which doesn't seem to be the case.
>> >>
>> >> Cosmin
>> >>
>> >> On Mar 18, 2010, at 8:57 PM, Andrew Purtell wrote:
>> >>
>> >> > +1
>> >> >
>> >> > Thanks, this means I can get off of branch for dev and move back to
>> >> trunk.
>> >> >
>> >> >  - Andy
>> >> >
>> >> >> From: Stack <[EMAIL PROTECTED]>
>> >> >> Subject: [MORE DISCUSSION] Moving hbase trunk (temporarily) back to
>> >> hadoop  0.20.x (from head of the hadoop 0.21 branch)
>> >> >> To: "HBase Dev List" <[EMAIL PROTECTED]>
>> >> >> Date: Thursday, March 18, 2010, 11:28 AM
>> >> >>
>> >> >> Unless there is objection, in the next day or so, I'd like
>> >> >> to revert the hadoop that is in hbase trunk and replace it
>> >> >> with hadoop 0.20.2.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >
>> >
>>
>
>
>
> --
> Connect to me at http://www.facebook.com/dhruba
>