I was reading up on HBase Replication and wanted to make sure I'm not
missing something.

Given that replication happens asynchronously the replication strategy has
an "eventually consistent" policy.

I was considering using this feature for Production / Disaster Recovery
setup.

Is there a way to enforce Consistency so that if my PROD environment should
ever go down, I can 100% sure that DR will be completely up to date?

Thank you,
Juan

What failure condition are you trying to safeguard against? A full
data center failure? That's when you would lose your entire cluster
and need the DR to kick in. Otherwise, you could deploy such that an
entire rack failure or even a row failure won't take you down. Just
span across multiple racks and rows. For a full data center failure,
the current replication scheme only gives you asynchronous replication
for reasons Ian mentioned.

On Dec 7, 2012, at 10:15 AM, "Juan P." <[EMAIL PROTECTED]> wrote:

> I was reading up on HBase Replication and wanted to make sure I'm not
> missing something.
>
> Given that replication happens asynchronously the replication strategy has
> an "eventually consistent" policy.
>
> I was considering using this feature for Production / Disaster Recovery
> setup.
>
> Is there a way to enforce Consistency so that if my PROD environment should
> ever go down, I can 100% sure that DR will be completely up to date?
>
> Thank you,
> Juan

Juan,

No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.

Ian

On Dec 7, 2012, at 12:14 PM, Juan P. wrote:

I was reading up on HBase Replication and wanted to make sure I'm not
missing something.

Given that replication happens asynchronously the replication strategy has
an "eventually consistent" policy.

I was considering using this feature for Production / Disaster Recovery
setup.

Is there a way to enforce Consistency so that if my PROD environment should
ever go down, I can 100% sure that DR will be completely up to date?

Thank you,
Juan

"Strongly consistent reads/writes: HBase is not an "eventually consistent" DataStore. This makes it very suitable for tasks such as high-speed counter aggregation"

http://hbase.apache.org/book/architecture.html
Am I missing something ?

- Sri

>________________________________
> From: Ian Varley <[EMAIL PROTECTED]>
>To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>Sent: Friday, 7 December 2012, 23:49
>Subject: Re: PROD/DR - Replication
>
>Juan,
>
>No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.
>
>Ian
>
>On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
>
>I was reading up on HBase Replication and wanted to make sure I'm not
>missing something.
>
>Given that replication happens asynchronously the replication strategy has
>an "eventually consistent" policy.
>
>I was considering using this feature for Production / Disaster Recovery
>setup.
>
>Is there a way to enforce Consistency so that if my PROD environment should
>ever go down, I can 100% sure that DR will be completely up to date?
>
>Thank you,
>Juan
>
>
>
>

Yes, I think so. A single HBase cluster can't (or, at least, really shouldn't) span multiple data centers; the strong consistency you refer to is only available within a cluster.

But the replication you were referring to in your initial email is cross-data center, between two or more clusters. That's where you can't get strong consistency.

Ian

On Dec 7, 2012, at 1:38 PM, "sriraam h" <[EMAIL PROTECTED]> wrote:

> "Strongly consistent reads/writes: HBase is not an "eventually consistent" DataStore. This makes it very suitable for tasks such as high-speed counter aggregation"
>
> http://hbase.apache.org/book/architecture.html
>
>
> Am I missing something ?
>
> - Sri
>
>
>
>> ________________________________
>> From: Ian Varley <[EMAIL PROTECTED]>
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Sent: Friday, 7 December 2012, 23:49
>> Subject: Re: PROD/DR - Replication
>>
>> Juan,
>>
>> No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.
>>
>> Ian
>>
>> On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
>>
>> I was reading up on HBase Replication and wanted to make sure I'm not
>> missing something.
>>
>> Given that replication happens asynchronously the replication strategy has
>> an "eventually consistent" policy.
>>
>> I was considering using this feature for Production / Disaster Recovery
>> setup.
>>
>> Is there a way to enforce Consistency so that if my PROD environment should
>> ever go down, I can 100% sure that DR will be completely up to date?
>>
>> Thank you,
>> Juan
>>
>>
>>

Thanks Ian. I DID miss the point. The person who started the chain is a different person :)

- Sri

>________________________________
> From: Ian Varley <[EMAIL PROTECTED]>
>To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>Sent: Saturday, 8 December 2012, 1:21
>Subject: Re: PROD/DR - Replication
>
>Yes, I think so. A single HBase cluster can't (or, at least, really shouldn't) span multiple data centers; the strong consistency you refer to is only available within a cluster.
>
>But the replication you were referring to in your initial email is cross-data center, between two or more clusters. That's where you can't get strong consistency.
>
>Ian
>
>
>
>On Dec 7, 2012, at 1:38 PM, "sriraam h" <[EMAIL PROTECTED]> wrote:
>
>> "Strongly consistent reads/writes: HBase is not an "eventually consistent" DataStore. This makes it very suitable for tasks such as high-speed counter aggregation"
>>
>> http://hbase.apache.org/book/architecture.html
>>
>>
>> Am I missing something ?
>>
>> - Sri
>>
>>
>>
>>> ________________________________
>>> From: Ian Varley <[EMAIL PROTECTED]>
>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>> Sent: Friday, 7 December 2012, 23:49
>>> Subject: Re: PROD/DR - Replication
>>>
>>> Juan,
>>>
>>> No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.
>>>
>>> Ian
>>>
>>> On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
>>>
>>> I was reading up on HBase Replication and wanted to make sure I'm not
>>> missing something.
>>>
>>> Given that replication happens asynchronously the replication strategy has
>>> an "eventually consistent" policy.
>>>
>>> I was considering using this feature for Production / Disaster Recovery
>>> setup.
>>>
>>> Is there a way to enforce Consistency so that if my PROD environment should
>>> ever go down, I can 100% sure that DR will be completely up to date?
>>>
>>> Thank you,
>>> Juan
>>>
>>>
>>>
>
>
>

Ha - that's what I get for trying to answer list emails from my phone. :)

Ian

On Dec 7, 2012, at 1:58 PM, sriraam h wrote:

Thanks Ian. I DID miss the point. The person who started the chain is a different person :)

- Sri

________________________________
From: Ian Varley <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Sent: Saturday, 8 December 2012, 1:21
Subject: Re: PROD/DR - Replication

Yes, I think so. A single HBase cluster can't (or, at least, really shouldn't) span multiple data centers; the strong consistency you refer to is only available within a cluster.

But the replication you were referring to in your initial email is cross-data center, between two or more clusters. That's where you can't get strong consistency.

Ian

On Dec 7, 2012, at 1:38 PM, "sriraam h" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

"Strongly consistent reads/writes: HBase is not an "eventually consistent" DataStore. This makes it very suitable for tasks such as high-speed counter aggregation"

http://hbase.apache.org/book/architecture.html
Am I missing something ?

- Sri

________________________________
From: Ian Varley <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Sent: Friday, 7 December 2012, 23:49
Subject: Re: PROD/DR - Replication

Juan,

No; that would mean every single write to HBase has to wait for an ACK from a remote data center, which would decrease your cluster throughput dramatically. If you need that, consider other database solutions.

Ian

On Dec 7, 2012, at 12:14 PM, Juan P. wrote:

I was reading up on HBase Replication and wanted to make sure I'm not
missing something.

Given that replication happens asynchronously the replication strategy has
an "eventually consistent" policy.

I was considering using this feature for Production / Disaster Recovery
setup.

Is there a way to enforce Consistency so that if my PROD environment should
ever go down, I can 100% sure that DR will be completely up to date?

Thank you,
Juan

Agree with what Ian says below except I'd say with work it's possible to do
cross DC quorum writes in a layer on top of HBase that persists into
HBase, like
when Google built Megastore on BigTable. At this year's VLDB if I recall
correctly there was a megastore-like system with an interesting variation built
on HBase presented in http://arxiv.org/abs/1208.0270 .

On Saturday, December 8, 2012, Ian Varley wrote:

> Juan,
>
> No; that would mean every single write to HBase has to wait for an ACK
> from a remote data center, which would decrease your cluster throughput
> dramatically. If you need that, consider other database solutions.
>
> Ian
>
> On Dec 7, 2012, at 12:14 PM, Juan P. wrote:
>
> I was reading up on HBase Replication and wanted to make sure I'm not
> missing something.
>
> Given that replication happens asynchronously the replication strategy has
> an "eventually consistent" policy.
>
> I was considering using this feature for Production / Disaster Recovery
> setup.
>
> Is there a way to enforce Consistency so that if my PROD environment should
> ever go down, I can 100% sure that DR will be completely up to date?
>
> Thank you,
> Juan
>
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)