Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Zookeeper ensemble backup questions?


Copy link to this message
-
Re: Zookeeper ensemble backup questions?
thanks, Sergey!

On Fri, Jul 19, 2013 at 11:38 AM, Sergey Maslyakov <[EMAIL PROTECTED]> wrote:
> Alexander,
>
> I cannot speak to that intelligently. I don't know implementation and
> design details well enough. However, a snapshot of this kind, if it is
> produced by ZooKeeperServer.takeSnapshot() method, should contain not just
> the DataTree content but the sessions too. I am not sure if this covers
> ephemeral nodes though but I am pretty sure this is way Zookeeper server
> takes snapshots driven by the transaction count.
>
>
> Regards,
> /Sergey
>
>
> On Fri, Jul 19, 2013 at 1:26 PM, Alexander Shraer <[EMAIL PROTECTED]> wrote:
>
>> Sergey,
>>
>> Can you explain what happens with ephemeral nodes during snapshot and
>> restore ?
>>
>> Thanks,
>> Alex
>>
>> On Fri, Jul 19, 2013 at 11:21 AM, Sergey Maslyakov <[EMAIL PROTECTED]>
>> wrote:
>> > Jordan,
>> >
>> > It depends on the use case. Some use cases deal with 99% or more of
>> static
>> > non-ephemeral nodes. And this is why the Exhibitor approach does not work
>> > for those use cases.
>> >
>> >
>> > Regards,
>> > /Sergey
>> >
>> >
>> > On Fri, Jul 19, 2013 at 1:13 PM, Jordan Zimmerman <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> >> I'm a bit late to this one but I don't understand the need for complete
>> >> backup of ZK data. In my experience, 99% of ZNodes are ephemeral.
>> >> Therefore, it would be wrong to restore those nodes. In a disaster, the
>> >> connection sessions would expire and you would not want ephemeral nodes
>> >> restored. This is why in Exhibitor I took the approach of selective
>> restore.
>> >>
>> >> -Jordan
>> >>
>> >> On Jul 19, 2013, at 11:00 AM, jack ma <[EMAIL PROTECTED]> wrote:
>> >>
>> >> > I asked those question in the thread
>> >> >
>> >>
>> http://mail-archives.apache.org/mod_mbox/zookeeper-user/201307.mbox/%3cCAB+cfdwhOV0JfB04=MpO_+i-4ou=VbL=EG2XS557+j+[EMAIL PROTECTED]%3e
>> >> ,
>> >> > but there is no response for that.
>> >> >
>> >> > So I posted those questions again here, hopefully I could get helps
>> >> > from the community.
>> >> >
>> >> > I want to make sure I am fully understanding the procedures of
>> zookeeper
>> >> > backup and disaster recovery:
>> >> >
>> >> > For the backup procedures at zookeeper assemble:
>> >> > (1) Login to any host which state is "Serving"
>> >> >           Question:
>> >> >                  Do I have to login to leader node, or any node is ok?
>> >> > (2) Copy latest snapshot file and transaction log from version-2
>> >> directory.
>> >> >           Question:
>> >> >                  How to make sure we do not copy corrupt files if the
>> >> > snapshot/transaction log is in the middle of update? Do we have to
>> >> shutdown
>> >> > the node to make the copy?
>> >> >                  besides the transaction log and snapshot, do we have
>> to
>> >> > copy other files such as the ecoch files
>> >> >
>> >> > For the disaster recovery procedures at zookeeper assemble:
>> >> > (1) recreate the machines for the zookeeper ensemble
>> >> > (2) copy snapshot/transaction log we backed up into the zookeeper
>> >> > dataDir\version-2 and logDir\version2.
>> >> >           Question:
>> >> >                 Do we have to copy the epoch files?
>> >> >                 Do we have to copy snapshot/transaction log backed up
>> to
>> >> > all the zookeeper node, or just the first node we starts?
>> >> >
>> >> > Appreciate your time and help.
>> >> > Jack
>> >>
>> >>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB