Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - Re-init Accumulo over existing installation


Copy link to this message
-
Re: Re-init Accumulo over existing installation
Krishmin Rai 2012-07-05, 18:06
Keith, I created Accumulo-671 about a ZK re-initialization process.

John: definitely not using /tmp as the ZK data dir, but if no one else has experienced this kind of data loss with ZK, I'm ready to chalk it up to some kind of one-time user error on our part. If it happens again, I'll dig in further.

Thanks for the quick responses!
Krishmin

On Jul 5, 2012, at 11:35 AM, Keith Turner wrote:

> I have never seen this happen, I have found zookeeper to be very
> reliable.  I think Accumulo needs a utility to handle this case of
> reinitializing just zookeeper.  Would you like to open a ticket?
>
> Zookeeper does store some important persistent info, like mappings of
> table names to table ids, table config, users data, and FATE ops.
>
>
> On Thu, Jul 5, 2012 at 11:15 AM, Krishmin Rai <[EMAIL PROTECTED]> wrote:
>> Hi All,
>>  We've recently encountered a strange situation on a small test cluster: after an awkward crash, our ZooKeeper data was erased and we no longer have the [accumulo] znode. The HDFS accumulo directory is intact, so all the RFiles and etc are still there, but it's not clear how best to bring Accumulo back up to its previous state. Obviously just starting Accumulo as-is complains about the missing znode ("Waiting for accumulo to be initialized"), whereas re-initializing is not possible over existing HDFS directories ("It appears this location was previously initialized, exiting").
>>
>>  A couple of questions about recovery strategies:
>>
>> 1) Is there any way to re-create the znode for a previous instance-id? My understanding is that ZK is mostly used to store ephemeral data (such as which tserver is currently responsible for which tablets) and things like users (which we could re-create), so perhaps this is plausible?
>>
>> 2) I imagine that I could init Accumulo with a new instance.dfs.dir, then import the RFiles from the old installation back in. I see Patrick just asked a related question, so, with the data integrity caveats, I would essentially be following the last of the steps in ACCUMULO-456.
>>
>> 3) This is a vague question, but have any of you had experience with the [accumulo] znode being entirely deleted? Aside from stopping/starting ZK (3.3.5) and Accumulo 1.4.0 (possibly with a force-quit), I'm not sure what we could have done to actually delete it.
>>
>> This is just a test instance, and the data could easily be recreated, but I want to take this opportunity to learn a little more about Accumulo plumbing and maintenance.
>>
>> Thanks,
>> Krishmin
>>
>>
>>
>>