Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Backups


+
Jordan Zimmerman 2012-01-13, 22:24
+
Camille Fournier 2012-01-13, 22:28
+
Jordan Zimmerman 2012-01-13, 22:29
+
Patrick Hunt 2012-01-17, 01:39
+
Jordan Zimmerman 2012-01-17, 01:42
+
Camille Fournier 2012-01-17, 01:49
+
Neha Narkhede 2012-01-17, 18:10
+
Jordan Zimmerman 2012-01-17, 19:06
+
Jordan Zimmerman 2012-01-18, 00:38
Since you started this thread, I've been thinking about the idea of  
backing up, and I'm not sure I understand the motivation and if it is  
ok to violate safety properties.

Given that ZooKeeper is used for coordination, I would think that in  
many cases all its state can be reconstructed in an algorithmic  
manner. Perhaps the use case for a backup would be the one in which it  
is being used as a database, for example, to keep the metadata of a  
file system. Periodic backups or even keeping an observer, however,  
won't guarantee that if you bring the system up using that backup  
you'll have all committed operations. The state of the leader reflects  
all committed operations, but one needs to have the latest state of  
the transaction log to not miss an update.

But, it is true that I'm assuming that you can't miss updates. If you  
can miss updates, then that's a different story. By missing updates  
we'll be violating durability, which is  a property that ZooKeeper is  
supposed to provide, so I'm trying to understand in which cases  
violating durability would be acceptable. If it is not acceptable and  
you still want to have a backup, then I don't see a way other than  
shutting down the clients before you take a backup, which doesn't seem  
to be what is being proposed here.

-Flavio
On Jan 18, 2012, at 1:38 AM, Jordan Zimmerman wrote:

> Neha - can you send me your email address. Send it to:
> [EMAIL PROTECTED]
>
> On 1/17/12 10:10 AM, "Neha Narkhede" <[EMAIL PROTECTED]> wrote:
>
>> Jordan,
>>
>> I'd be interested in previewing it. Let me know.
>>
>> Thanks,
>> Neha
>>
>> On Mon, Jan 16, 2012 at 5:42 PM, Jordan Zimmerman
>> <[EMAIL PROTECTED]> wrote:
>>> We'll be backing up to S3. Wouldn't it be redundant to backup all  
>>> the
>>> instances?
>>>
>>> -JZ
>>>
>>> P.S. I'm working on a ZooKeeper instance manager that will have
>>> backup/restore and a bunch of other stuff. We'll be open sourcing  
>>> it. If
>>> anyone is interested in previewing it let me know.
>>>
>>>
>>> On 1/16/12 5:39 PM, "Patrick Hunt" <[EMAIL PROTECTED]> wrote:
>>>
>>>> Why would you limit to the leader? Wouldn't backing up any server  
>>>> (as
>>>> long as it's active) be sufficient? If you search the list it's  
>>>> been
>>>> discussed before, using Observers seemed like a reasonable option  
>>>> as
>>>> well.
>>>>
>>>> Patrick
>>>>
>>>> On Fri, Jan 13, 2012 at 2:29 PM, Jordan Zimmerman
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> That's easy as the backup app is running on the same machine as  
>>>>> the ZK
>>>>> instance. I can use 'stat' to see if "my" instance is the leader.
>>>>>
>>>>> On 1/13/12 2:28 PM, "Camille Fournier" <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> You want to have to figure out who the leader is every time you  
>>>>>> want
>>>>>> to
>>>>>> take a backup? That would be the downside to this strategy I  
>>>>>> would
>>>>>> think.
>>>>>>
>>>>>> C
>>>>>>
>>>>>> From my phone
>>>>>> On Jan 13, 2012 5:24 PM, "Jordan Zimmerman" <[EMAIL PROTECTED]
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>>> As a backup strategy, it seems I would only want to backup  
>>>>>>> snapshots
>>>>>>> from
>>>>>>> the leader. Does that make sense?
>>>>>>>
>>>>>>> -JZ
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
>

flavio
junqueira

research scientist

[EMAIL PROTECTED]
direct +34 93-183-8828

avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300    fax (408) 349 3301

+
Jordan Zimmerman 2012-01-19, 17:32
+
Ted Dunning 2012-01-19, 18:11
+
Jordan Zimmerman 2012-01-19, 18:16
+
Ted Dunning 2012-01-19, 18:23
+
Patrick Hunt 2012-01-19, 18:24
+
Ted Dunning 2012-01-19, 19:40
+
Flavio Junqueira 2012-01-19, 18:39
+
Jordan Zimmerman 2012-01-19, 19:07
+
Flavio Junqueira 2012-01-19, 19:30
+
Jordan Zimmerman 2012-01-19, 19:32
+
Ted Dunning 2012-01-19, 19:42
+
kishore g 2012-01-20, 07:42
+
Patrick Hunt 2012-01-20, 17:01
+
Ted Dunning 2012-01-19, 19:41
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB