Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> forceSync=no


+
Raj N 2012-06-14, 14:56
+
Jonathan Simms 2012-06-14, 17:03
+
Raj N 2012-06-14, 18:56
+
Patrick Hunt 2012-06-15, 18:17
+
Raj N 2012-06-15, 19:45
+
Patrick Hunt 2012-06-16, 00:33
+
Flavio Junqueira 2012-06-16, 11:25
+
Raj N 2012-06-16, 16:05
+
Flavio Junqueira 2012-06-18, 14:41
Copy link to this message
-
Re: forceSync=no
Raj,
 Are you seeing any perf improvements by using that setting? I had run
some benchmarks and didnt find much improvement even if I did
forceSync = no.

thanks
mahadev

On Fri, Jun 15, 2012 at 12:45 PM, Raj N <[EMAIL PROTECTED]> wrote:
> Thanks Patrick. So back to my original question. Will forceSync=no cause a
> zookeeper node to not start up cleanly if it crashed? It doesn't happen all
> the time. But it happened to me once. I had to cleanup all the transaction
> logs on all the nodes and then bring the nodes back up. For my use case I
> can tolerate losing data in the worst case scenario since I can regenerate
> all the data. But I should be able to tolerate a single point of failure as
> long as the node comes back up cleanly. Can zookeeper recover from a
> corrupt transaction log using existing snapshots and then replaying
> messages from its peers?
>
> Thanks
> -Raj
>
> On Fri, Jun 15, 2012 at 2:17 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
>> For a single node failure I believe so.
>>
>> Patrick
>>
>> On Thu, Jun 14, 2012 at 11:56 AM, Raj N <[EMAIL PROTECTED]> wrote:
>> > Sorry, I should have been more specific. By corrupt, I mean that the
>> > zookeeper node doesn't come back up on a restart. I would have imagined
>> > that zookeeper would sync the lost transactions from its peers. I agree I
>> > will have a problem if I have multiple failures. But for a single node
>> > failure in a 3-node ensemble, I should be able to recover even if
>> > forceSync=no.
>> >
>> > Thanks
>> > -Raj
>> >
>> > On Thu, Jun 14, 2012 at 1:03 PM, Jonathan Simms <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> There's a big warning in the documentation that says that's a
>> possibility.
>> >> If you don't force both Java and the OS to flush their IO buffers to
>> disk,
>> >> then you have no guarantees that your data is consistent.
>> >>
>> >> On 6/14/12 10:56 AM, "Raj N" <[EMAIL PROTECTED]> wrote:
>> >>
>> >> >Are you guys aware of any issues with forceSync=no that could cause the
>> >> >transaction log to get corrupted on a zookeeper crash.
>> >> >
>> >> >Thanks
>> >> >-Raj
>> >>
>> >>
>> >>
>>
+
Patrick Hunt 2012-06-16, 00:27
+
Vitalii Tymchyshyn 2012-06-18, 12:55