Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal


Copy link to this message
-
Re: [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal
The Accumulo implementation of the WAL is a separate set of daemons.
When you write to the WAL, you send your transactions to three of the
logging servers. When you do a recovery, I believe one of the three
servers that has the WAL for the down server copies it to HDFS and
then a MapReduce job splits the log and re-inserts the recovered data.
You should have the same survivability that you get with HDFS.

-Joey

On Mon, Sep 5, 2011 at 5:06 PM, Bill <[EMAIL PROTECTED]> wrote:
> On 04/09/11 07:43, Mathias Herberts wrote:
>>
>> On Sep 4, 2011 1:39 AM, "Bill de hÓra"<[EMAIL PROTECTED]>  wrote:
>>>
>>> On 02/09/11 19:06, Stack wrote:
>>>>
>>>> What do folks think?
>>>
>>>
>>> Not putting the log into hdfs seems like a good idea.
>>
>> I was somehow thinking the opposite as it makes irrecoverable machine
>> failures much more problematic. What makes you say it's a good idea?
>>
>
> Allows more control over the write path, specifically sequential I/O and
> crash recovery. Granted the commit needs to be replicated, but you need that
> regardless. Thinking a bit more it might not square with the regionserver
> model anyway, plus the Accumulo proposal mentions a service rather than a
> local disk. The WAL seems to be hardened up these days anyway making things
> like https://issues.apache.org/jira/browse/HBASE-4107 more of an edge case..
>
> Bill
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB