Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # dev >> Hadoop 2.0 Support for Accumulo 1.4 Branch


+
Sean Busbey 2013-10-14, 16:55
+
Josh Elser 2013-10-15, 03:02
+
Sean Busbey 2013-10-15, 03:45
+
Mike Drob 2013-10-15, 02:24
+
Sean Busbey 2013-10-15, 03:37
+
Ted Yu 2013-10-15, 02:29
+
Billie Rinaldi 2013-10-15, 03:57
+
dlmarion@... 2013-10-15, 12:16
+
Sean Busbey 2013-10-15, 15:16
+
Sean Busbey 2013-10-15, 15:20
+
Joey Echeverria 2013-10-15, 15:28
+
Sean Busbey 2013-10-18, 05:29
+
Sean Busbey 2013-11-12, 14:22
+
Josh Elser 2013-11-12, 19:12
+
Sean Busbey 2013-11-12, 20:24
+
Josh Elser 2013-11-12, 20:48
+
Sean Busbey 2013-11-12, 21:26
+
Josh Elser 2013-11-12, 21:49
Copy link to this message
-
Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
A user of 1.4.a should be able to move to 1.4.b without any "major"
infrastructure changes, such as swapping out HDFS or installing extra
add-ons.

I don't find much merit in debating local WAL vs HDFS WAL cost/benefit
since the only quantifiable evidence we have supported the move.

I should note, Sean, that if you see merit in the work, you don't need
community approval for forking and sharing. However, I do not think it is
in the community's best interest to continue to upgrade 1.4.

On Tue, Nov 12, 2013 at 2:12 PM, Josh Elser <[EMAIL PROTECTED]> wrote:

>
>> Based on recent feedback on ACCUMULO-1792 and ACCUMULO-1795, I want to
>> resurrect this thread to make sure everyone's concerns are addressed.
>>
>> For context, here's a link to the start of the last thread:
>>
>> http://bit.ly/1aPqKuH
>>
>>  From ACCUMULO-1792, ctubbsii:
>>
>>  I'd be reluctant to support any Hadoop 2.x support in the 1.4 release
>>>
>> line that breaks compatibility with 0.20. I don't think breaking 0.20
>>
>>> and then possibly fixing it again as a second step is acceptable (because
>>>
>> that subsequent work may not ever be done, and I don't think
>>
>>> we should break the compatibility contract that we've established with
>>>
>> 1.4.0).
>>
>> Chris, I believe keeping all of the work in a branch under the umbrella
>> jira of ACCUMULO-1790 will ensure that we don't end up with a 1.4 release
>> that doesn't have proper support for 0.20.203.
>>
>> Is there something beyond making sure the branch passes a full set of
>> release tests on 0.20.203 that you'd like to see? In the event that the
>> branch only ever contains the work for adding Hadoop 2, it's a simple
>> matter to abandon without rolling into the 1.4 development line.
>>
>>  From ACCUMULO-1795, bills (and +1ed by elserj and ctubbsii):
>>
>>  I'm very uncomfortable with risking breaking continuity in such an old
>>>
>> release, and I don't think managing two lines of 1.4 releases is
>>
>>> worth the effort. Though we have no official EOL policy, 1.3 was
>>>
>> practically dead in the water once 1.4 was around, and I hope we start
>>
>>> encouraging more adoption of 1.5 (and soon 1.6) versus continually
>>>
>> propping up 1.4.
>>
>> I'd love to get people to move off of 1.4. However, I think adding Hadoop
>> 2
>> support to 1.4 encourages this more than leaving it out.
>>
>
> I'm not sure I agree that adding Hadoop2 support to 1.4 encourages people
> to upgrade Accumulo. My gut reaction would be that it allows people to
> completely ignore Accumulo updates (ignoring moving to 1.4.5 which would
> allow them to do hadoop2 with your proposed changes)
>
>
>  Accumulo 1.5.x places a higher burden on HDFS than 1.4 did, and I'm not
>> surprised people find relying on 0.20 for the 1.5 WAL intimidating.
>> Upgrading both HDFS and Accumulo across major versions at once is asking
>> them to take on a bunch of risk. By adding in Hadoop 2 support to 1.4 we
>> allow them to break the risk up into steps: they can upgrade HDFS versions
>> first, get comfortable, then upgrade Accumulo to 1.5.
>>
>
> Personally, maintaining 0.20 compatibility is not a big concern on my
> radar. If you're still running an 0.20 release, I'd *really* hope that you
> have an upgrade path to 1.2.x (if not 2.2.x) scheduled.
>
> I think claiming that 1.5 has a higher burden on 1.4 is a bit of a
> fallacy. There were many problems and pains regarding WALs in <=1.4 that
> are very difficult to work with in a large environment (try finding WALs in
> server failure cases). I think the increased I/O on HDFS is a much smaller
> cost than the completely different I/O path that the old loggers have.
>
> I also think upgrading Accumulo is much less scary than upgrading HDFS,
> but that's just me.
>
> To me, it seems like the argument may be coming down to whether or not we
> break 0.20 hadoop compatibility on a bug-fix release and how concerned we
> are about letting users lag behind the upstream development.
>
>
>  I think the existing tickets under the umbrella of ACCUMULO-1790 should
+
Sean Busbey 2013-11-12, 20:28
+
William Slacum 2013-11-12, 21:14
+
William Slacum 2013-11-12, 21:24
+
Sean Busbey 2013-11-12, 21:49
+
Christopher 2013-11-15, 00:36
+
Christopher 2013-11-15, 00:27
+
Sean Busbey 2013-11-15, 00:39
+
Christopher 2013-11-15, 01:08
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB