Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Add Append-HBase support in upcoming 20.205


Copy link to this message
-
Re: Add Append-HBase support in upcoming 20.205
> From: Arun C Murthy <[EMAIL PROTECTED]>

> +1
>
> I think it's high time (I thought so too last Dec too:
> http://s.apache.org/jr) we had an official Hadoop release which supports
> HBase. Thanks for all the effort Andrew - it will be really nice to have
> an Apache 0.20 with security+append!

Thanks to Dhruba, Todd, Hairong, and the other original contributors of append support to HDFS 0.20.x.

I'm +1 obviously. :-)
Best regards,
    - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Arun C Murthy <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc:
> Sent: Friday, September 2, 2011 6:36 AM
> Subject: Re: Add Append-HBase support in upcoming 20.205
>
>
> On Aug 31, 2011, at 11:41 AM, sanjay Radia wrote:
>
>>
>>  I propose that the 20-append patches (details below)  be included in 20.205
> which will become the first official Apache
>>  release of Hadoop that supports Append and HBase.
>>
>
>>  Background:
>>  There hasn't been a official Apache release that supports HBase.
>>  The HBase community have instead been using the 20-append branch; the
> patches were contributed by the HBase community including Facebook. The Cloudera
> distribution has also included these patches.
>>  Andrew Purtell has ported these patches to 20-security branch.
>>
>
>
> +1
>
> I think it's high time (I thought so too last Dec too:
> http://s.apache.org/jr) we had an official Hadoop release which supports HBase.
> Thanks for all the effort Andrew - it will be really nice to have an Apache 0.20
> with security+append!
>
> Arun
>
>>  Risk Level:
>>  These patches have been used and tested on large HBase clusters by FB , by
> those who use 20-append branch directly (various users including a 500 node
> HBase cluster at Yahoo) and by those that use the Cloudera distribution. We have
> reviewed the patches and have conducted further tests; testing and validation
> continues.
>>
>>
>>  Patches:
>>  HDFS-200. Support append and sync for hadoop 0.20 branch.
>>  HDFS-142. Blocks that are being written by a client are stored in the
> blocksBeingWritten directory.
>>  HDFS-1057.  Concurrent readers hit ChecksumExceptions if following a writer
> to very end of file
>>  HDFS-724.  Use a bidirectional heartbeat to detect stuck pipeline.
>>  HDFS-895. Allow hflush/sync to occur in parallel with new writes to the
> file.
>>  HDFS-1520. Lightweight NameNode operation recoverLease to trigger lease
> recovery.
>>  HDFS-1555. Disallow pipelien recovery if a file is already being lease
> recovered.
>>  HDFS-1554. New semantics for recoverLease.
>>  HDFS-988. Fix bug where savenameSpace can corrupt edits log.
>>  HDFS-826. Allow a mechanism for an application to detect that datanode(s)
> have died in the write pipeline.
>>  HDFS-630. Client can exclude specific nodes in the write pipeline.
>>  HDFS-1141. completeFile does not check lease ownership.
>>  HDFS-1204. Lease expiration should recover single files, not entire lease
> holder
>>  HDFS-1254. Support append/sync via the default configuration.
>>  HDFS-1346. DFSClient receives out of order packet ack.
>>  HDFS-1054. remove sleep before retry for allocating a block.
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB