Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> .oldlogs in region dir


Copy link to this message
-
Re: .oldlogs in region dir
Hi Dhruba,

Yes, but that is the one on the root dir level. I am referring to the one on
the region level.

Cheers,
Lars

On Tue, Jul 5, 2011 at 6:25 AM, Dhruba Borthakur <[EMAIL PROTECTED]> wrote:

> Doesn't the wide-area HBase replication use the .oldlogs to keep the slave
> hBase cluster in sync?
>
> thanks
> dhruba
>
> On Tue, Jul 5, 2011 at 7:27 AM, Lars George <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > Ah, I see Ted has that also questioned in HBASE-4010. Good.
> >
> > And I was slightly wrong below, as there is a .logs and .oldlogs for the
> > newly created regions. I agree that when there is no chance for the
> region
> > to take edits before it gets the proper log, then we should drop this.
> Those
> > dangling dirs do not harm, but also do not help either.
> >
> > Lars
> >
> > On Jul 5, 2011, at 8:39 AM, Lars George wrote:
> >
> > > Hi,
> > >
> > > Could someone confirm that we dropped the replay of the .oldlogs in the
> > region directory because it is not needed anymore? Before the master
> rewrite
> > we were, just as now, creating the region and for a brief period it had
> its
> > own log directory, the .oldlogs under the region directory. The
> > HRegion.initialize() would replay it in case an edit to the region
> arrived
> > in between. It would then also remove it:
> > >
> > >   public void initialize(Path initialFiles, final Progressable
> reporter)
> > >   throws IOException {
> > >     Path oldLogFile = new Path(regiondir, HREGION_OLDLOGFILE_NAME);
> > > ...
> > >     // Play log if one.  Delete when done.
> > >     doReconstructionLog(oldLogFile, minSeqIdToRecover, maxSeqId,
> > reporter);
> > >     if (fs.exists(oldLogFile)) {
> > >       if (LOG.isDebugEnabled()) {
> > >         LOG.debug("Deleting old log file: " + oldLogFile);
> > >       }
> > >       fs.delete(oldLogFile, false);
> > >     }
> > > ...
> > >
> > > Now we still create that directory, but the above code has been
> dropped.
> > Only the recovered.edits is replayed instead while the .oldlogs now hang
> > around. I am not even sure when they may contain an entry, but if they
> don't
> > then we can either do not use "<regiondir>/.oldlogs", but point it to the
> > new "<regiondir>/.tmp" - or we can add a unconditional delete in the
> > initialize as before.
> > >
> > > Create a JIRA for this?
> > >
> > > Lars
> >
> >
>
>
> --
> Connect to me at http://www.facebook.com/dhruba
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB