Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - .oldlogs in region dir


Copy link to this message
-
Re: .oldlogs in region dir
Lars George 2011-07-05, 14:41
Hi Dhruba,

Yes, but that is the one on the root dir level. I am referring to the one on
the region level.

Cheers,
Lars

On Tue, Jul 5, 2011 at 6:25 AM, Dhruba Borthakur <[EMAIL PROTECTED]> wrote:

> Doesn't the wide-area HBase replication use the .oldlogs to keep the slave
> hBase cluster in sync?
>
> thanks
> dhruba
>
> On Tue, Jul 5, 2011 at 7:27 AM, Lars George <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > Ah, I see Ted has that also questioned in HBASE-4010. Good.
> >
> > And I was slightly wrong below, as there is a .logs and .oldlogs for the
> > newly created regions. I agree that when there is no chance for the
> region
> > to take edits before it gets the proper log, then we should drop this.
> Those
> > dangling dirs do not harm, but also do not help either.
> >
> > Lars
> >
> > On Jul 5, 2011, at 8:39 AM, Lars George wrote:
> >
> > > Hi,
> > >
> > > Could someone confirm that we dropped the replay of the .oldlogs in the
> > region directory because it is not needed anymore? Before the master
> rewrite
> > we were, just as now, creating the region and for a brief period it had
> its
> > own log directory, the .oldlogs under the region directory. The
> > HRegion.initialize() would replay it in case an edit to the region
> arrived
> > in between. It would then also remove it:
> > >
> > >   public void initialize(Path initialFiles, final Progressable
> reporter)
> > >   throws IOException {
> > >     Path oldLogFile = new Path(regiondir, HREGION_OLDLOGFILE_NAME);
> > > ...
> > >     // Play log if one.  Delete when done.
> > >     doReconstructionLog(oldLogFile, minSeqIdToRecover, maxSeqId,
> > reporter);
> > >     if (fs.exists(oldLogFile)) {
> > >       if (LOG.isDebugEnabled()) {
> > >         LOG.debug("Deleting old log file: " + oldLogFile);
> > >       }
> > >       fs.delete(oldLogFile, false);
> > >     }
> > > ...
> > >
> > > Now we still create that directory, but the above code has been
> dropped.
> > Only the recovered.edits is replayed instead while the .oldlogs now hang
> > around. I am not even sure when they may contain an entry, but if they
> don't
> > then we can either do not use "<regiondir>/.oldlogs", but point it to the
> > new "<regiondir>/.tmp" - or we can add a unconditional delete in the
> > initialize as before.
> > >
> > > Create a JIRA for this?
> > >
> > > Lars
> >
> >
>
>
> --
> Connect to me at http://www.facebook.com/dhruba
>