Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal


Copy link to this message
-
Re: [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal
Joey Echeverria 2011-09-05, 21:35
The Accumulo implementation of the WAL is a separate set of daemons.
When you write to the WAL, you send your transactions to three of the
logging servers. When you do a recovery, I believe one of the three
servers that has the WAL for the down server copies it to HDFS and
then a MapReduce job splits the log and re-inserts the recovered data.
You should have the same survivability that you get with HDFS.

-Joey

On Mon, Sep 5, 2011 at 5:06 PM, Bill <[EMAIL PROTECTED]> wrote:
> On 04/09/11 07:43, Mathias Herberts wrote:
>>
>> On Sep 4, 2011 1:39 AM, "Bill de hÓra"<[EMAIL PROTECTED]>  wrote:
>>>
>>> On 02/09/11 19:06, Stack wrote:
>>>>
>>>> What do folks think?
>>>
>>>
>>> Not putting the log into hdfs seems like a good idea.
>>
>> I was somehow thinking the opposite as it makes irrecoverable machine
>> failures much more problematic. What makes you say it's a good idea?
>>
>
> Allows more control over the write path, specifically sequential I/O and
> crash recovery. Granted the commit needs to be replicated, but you need that
> regardless. Thinking a bit more it might not square with the regionserver
> model anyway, plus the Accumulo proposal mentions a service rather than a
> local disk. The WAL seems to be hardened up these days anyway making things
> like https://issues.apache.org/jira/browse/HBASE-4107 more of an edge case..
>
> Bill
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434