Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Clearing the .tmp directory on master startup


Copy link to this message
-
Re: Clearing the .tmp directory on master startup
right, I was thinking at the delete case..
so you want to finish the table creation...

but what is the message given to the user when the master goes down?
I think the client receive a failure, and the code it may expect the table
to be not there...
(aside from the fact that now the table is half there, and you've to run
hbck)

when this fix it will be implemented, you're saying to the client
something went wrong during the creation... but your table may be ready

I prefer to rollback the operation in case of failure (dropping everything)
but if you can recreate the whole table I guess the only problem you should
have to solve is documenting correctly the expected behaviour and having
a consistent message for the client.

On Mon, Mar 25, 2013 at 12:31 PM, ramkrishna vasudevan <
[EMAIL PROTECTED]> wrote:

> My plan is like this, suppose a master goes down just after creating the
> regioninfo files in the tmp folder then we can try to use that info for
> recreating the table regions on master restart.
>
> But one problem here could be that before creating all the regioninfos in
> the .tmp location if the master goes down we should be able to identify
> this scenario also.
>
> Regards
> Ram
>
> On Mon, Mar 25, 2013 at 5:38 PM, Matteo Bertozzi <[EMAIL PROTECTED]
> >wrote:
>
> > snapshots knows about the /hbase/.tmp directory but unless you remove the
> > files
> > without archiving or move to a location that is not the original one
> > everything is fine.
> >
> > for me is ok moving the .tmp deletion later.
> > do you want to restore the unfinished delete instead instead of finishing
> > the deletion?
> >
> > On Mon, Mar 25, 2013 at 11:55 AM, ramkrishna vasudevan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi  Devs
> > >
> > > Currently the HMaster initilization deletes the files under .tmp using
> > the
> > > HFileArchiver.
> > > in HMaster().new MasterFileSystem().createInitialFileSystemLayout().
> > >
> > > Actually i was thinking of using this info in the .tmp directory for
> the
> > > tables that were partially created so that on master restart i can use
> > this
> > > info to recreate the table fully.
> > > But i found that we are deleting this .tmp directory.
> > >
> > > My question here is will it be ok to delete the .tmp directory after
> > > the this.assignmentManager.joinCluster(), call in
> > > HMaster.finishInitialization?
> > >
> > > Because by this time i would be knowing what are the partially created
> > > tables.
> > >
> > > Just wanted to confirm before i can make this change?  My worry was,
> may
> > be
> > > snapshot is trying to utilize this info and doing something.
> > >
> > > Regards
> > > Ram
> > >
> >
>