Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # dev >> Hadoop 2.0 Support for Accumulo 1.4 Branch


+
Sean Busbey 2013-10-14, 16:55
+
Josh Elser 2013-10-15, 03:02
+
Sean Busbey 2013-10-15, 03:45
+
Mike Drob 2013-10-15, 02:24
+
Sean Busbey 2013-10-15, 03:37
+
Ted Yu 2013-10-15, 02:29
+
Billie Rinaldi 2013-10-15, 03:57
+
dlmarion@... 2013-10-15, 12:16
+
Sean Busbey 2013-10-15, 15:16
+
Sean Busbey 2013-10-15, 15:20
+
Joey Echeverria 2013-10-15, 15:28
+
Sean Busbey 2013-10-18, 05:29
+
Sean Busbey 2013-11-12, 14:22
+
Josh Elser 2013-11-12, 19:12
+
Sean Busbey 2013-11-12, 20:24
+
Josh Elser 2013-11-12, 20:48
+
Sean Busbey 2013-11-12, 21:26
+
Josh Elser 2013-11-12, 21:49
Copy link to this message
-
Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
A user of 1.4.a should be able to move to 1.4.b without any "major"
infrastructure changes, such as swapping out HDFS or installing extra
add-ons.

I don't find much merit in debating local WAL vs HDFS WAL cost/benefit
since the only quantifiable evidence we have supported the move.

I should note, Sean, that if you see merit in the work, you don't need
community approval for forking and sharing. However, I do not think it is
in the community's best interest to continue to upgrade 1.4.

On Tue, Nov 12, 2013 at 2:12 PM, Josh Elser <[EMAIL PROTECTED]> wrote:

>
>> Based on recent feedback on ACCUMULO-1792 and ACCUMULO-1795, I want to
>> resurrect this thread to make sure everyone's concerns are addressed.
>>
>> For context, here's a link to the start of the last thread:
>>
>> http://bit.ly/1aPqKuH
>>
>>  From ACCUMULO-1792, ctubbsii:
>>
>>  I'd be reluctant to support any Hadoop 2.x support in the 1.4 release
>>>
>> line that breaks compatibility with 0.20. I don't think breaking 0.20
>>
>>> and then possibly fixing it again as a second step is acceptable (because
>>>
>> that subsequent work may not ever be done, and I don't think
>>
>>> we should break the compatibility contract that we've established with
>>>
>> 1.4.0).
>>
>> Chris, I believe keeping all of the work in a branch under the umbrella
>> jira of ACCUMULO-1790 will ensure that we don't end up with a 1.4 release
>> that doesn't have proper support for 0.20.203.
>>
>> Is there something beyond making sure the branch passes a full set of
>> release tests on 0.20.203 that you'd like to see? In the event that the
>> branch only ever contains the work for adding Hadoop 2, it's a simple
>> matter to abandon without rolling into the 1.4 development line.
>>
>>  From ACCUMULO-1795, bills (and +1ed by elserj and ctubbsii):
>>
>>  I'm very uncomfortable with risking breaking continuity in such an old
>>>
>> release, and I don't think managing two lines of 1.4 releases is
>>
>>> worth the effort. Though we have no official EOL policy, 1.3 was
>>>
>> practically dead in the water once 1.4 was around, and I hope we start
>>
>>> encouraging more adoption of 1.5 (and soon 1.6) versus continually
>>>
>> propping up 1.4.
>>
>> I'd love to get people to move off of 1.4. However, I think adding Hadoop
>> 2
>> support to 1.4 encourages this more than leaving it out.
>>
>
> I'm not sure I agree that adding Hadoop2 support to 1.4 encourages people
> to upgrade Accumulo. My gut reaction would be that it allows people to
> completely ignore Accumulo updates (ignoring moving to 1.4.5 which would
> allow them to do hadoop2 with your proposed changes)
>
>
>  Accumulo 1.5.x places a higher burden on HDFS than 1.4 did, and I'm not
>> surprised people find relying on 0.20 for the 1.5 WAL intimidating.
>> Upgrading both HDFS and Accumulo across major versions at once is asking
>> them to take on a bunch of risk. By adding in Hadoop 2 support to 1.4 we
>> allow them to break the risk up into steps: they can upgrade HDFS versions
>> first, get comfortable, then upgrade Accumulo to 1.5.
>>
>
> Personally, maintaining 0.20 compatibility is not a big concern on my
> radar. If you're still running an 0.20 release, I'd *really* hope that you
> have an upgrade path to 1.2.x (if not 2.2.x) scheduled.
>
> I think claiming that 1.5 has a higher burden on 1.4 is a bit of a
> fallacy. There were many problems and pains regarding WALs in <=1.4 that
> are very difficult to work with in a large environment (try finding WALs in
> server failure cases). I think the increased I/O on HDFS is a much smaller
> cost than the completely different I/O path that the old loggers have.
>
> I also think upgrading Accumulo is much less scary than upgrading HDFS,
> but that's just me.
>
> To me, it seems like the argument may be coming down to whether or not we
> break 0.20 hadoop compatibility on a bug-fix release and how concerned we
> are about letting users lag behind the upstream development.
>
>
>  I think the existing tickets under the umbrella of ACCUMULO-1790 should
+
Sean Busbey 2013-11-12, 20:28
+
William Slacum 2013-11-12, 21:14
+
William Slacum 2013-11-12, 21:24
+
Sean Busbey 2013-11-12, 21:49
+
Christopher 2013-11-15, 00:36
+
Christopher 2013-11-15, 00:27
+
Sean Busbey 2013-11-15, 00:39
+
Christopher 2013-11-15, 01:08