Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # dev - Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0


Copy link to this message
-
Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0
David Medinets 2013-10-24, 11:04
Can an application work with parts in Hadoop 1 and parts in YARN? Could we
convert one component at a time to work inside YARN? For example, first
move monitor then tracer, etc. Where there any architectural tradeoffs made
for Hadoop 1 that no longer apply in Hadoop 2?
On Thu, Oct 24, 2013 at 12:58 AM, Josh Elser <[EMAIL PROTECTED]> wrote:

> Highlights from hadoop.apache.org
>
>     YARN - A general purpose resource management system for Hadoop to
> allow MapReduce and other other data processing frameworks and services
>     High Availability for HDFS
>     HDFS Federation
>     HDFS Snapshots
>     NFSv3 access to data in HDFS
>     Support for running Hadoop on Microsoft Windows
>     Binary Compatibility for MapReduce applications built on hadoop-1.x
>     Substantial amount of integration testing with rest of projects in the
> ecosystem
>
> Additionally, I believe there are substantial improvements in HDFS which
> should improve general performance for all filesystem ops.
>
>
> On 10/23/13 9:51 PM, William Slacum wrote:
>
>> There wasn't any discussions in those tickets as to what Hadoop 2 provides
>> Accumulo. If we're going to still support 1, then any new features only
>> possible with 2 have to become optional until we ditch support for 1. Is
>> there anything people have in mind, feature wise, that Hadoop 2 would help
>> with?
>>
>>
>> On Wed, Oct 23, 2013 at 7:05 PM, Josh Elser <[EMAIL PROTECTED]> wrote:
>>
>>  To ensure that we get broader community interaction than only on a Jira
>>> issue [1], I want to get community feedback about the version of Hadoop
>>> which the default, deployed Accumulo artifacts will be compiled against.
>>>
>>> Currently, Accumulo builds against a Hadoop-1 series release
>>> (1.5.1-SNAPSHOT and 1.6.0-SNAPSHOT build against 1.2.1, and 1.5.0 builds
>>> against 1.0.4). Last week, the Apache Hadoop community voted to release
>>> 2.2.0 as GA (general availability) -- in other words, the Apache Hadoop
>>> community is calling Hadoop-2.2.0 "stable".
>>>
>>> As has been discussed across various issues on Jira, this means a few
>>> different things for Accumulo. Most importantly, this serves as a
>>> recommendation by us that users should be trying to use Hadoop-2.2.0 with
>>> Accumulo 1.6.0. This does *not* mean that we do not support Hadoop1 ([2]
>>> 1.2.1 specifically). Hadoop-1 support would still be "guaranteed" by us
>>> for
>>> 1.6.0.
>>>
>>> - Josh
>>>
>>> [1] https://issues.apache.org/****jira/browse/ACCUMULO-1419<https://issues.apache.org/**jira/browse/ACCUMULO-1419>
>>> <http**s://issues.apache.org/jira/**browse/ACCUMULO-1419<https://issues.apache.org/jira/browse/ACCUMULO-1419>
>>> >
>>> [2] https://issues.apache.org/****jira/browse/ACCUMULO-1643<https://issues.apache.org/**jira/browse/ACCUMULO-1643>
>>> <http**s://issues.apache.org/jira/**browse/ACCUMULO-1643<https://issues.apache.org/jira/browse/ACCUMULO-1643>
>>> >
>>>
>>>
>>