Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Re: How to replace MetaUtils.ScannerListener?


Copy link to this message
-
Re: How to replace MetaUtils.ScannerListener?
Enis Söztutar 2013-03-22, 18:23
CopyTable and Snapshots are very useful for backup, while merge and split
are only relevant if the table is serving reads and writes. I can see
having some tools for helping with cases where HBase cannot recover, but
apart from that, normal Hbase operations should not need offline splits or
merges.

I am not suggesting to remove it now, but if we get online merges rock
solid, we can reevaluate.

Enis
On Fri, Mar 22, 2013 at 9:31 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Point taken.
>
> But I don't think previous discussion was about removing offline merge.
>
> On Fri, Mar 22, 2013 at 8:19 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>
> > Let me offer a counter argument.  The offline splitting code is still
> > present even though online splitting isn't the problem it used to be. We
> > actually added an extenal wal replayer even though we have wal replay in
> > our normal recovery path.  Copy table still exists even though snapshots
> > and snapshot export exists.  Would we consider removing these?
> >
> > IMO, until online merge has been proven and hardened (gone through few
> > releases and folk using) I don't feel comfortable removing it.
> >
> > Jon.
> >
> > On Thu, Mar 21, 2013 at 7:07 PM, Jean-Marc Spaggiari <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi Enis,
> > >
> > > I totaly agree. But even if online merge are available, maybe offline
> > > merges can still usefull in case the cluster is down for maintenance,
> > > or because there is any issue to start it, or anything else? We have
> > > it, so we should maybe try to keep it?
> > >
> > > JM
> > >
> > > 2013/3/21 Enis Söztutar <[EMAIL PROTECTED]>:
> > > > Thanks J-M.
> > > >
> > > > What I am trying to understand is that whether we should cut the cord
> > for
> > > > offline merge once online is working. If you think about it, there
> > should
> > > > not be a need to merge offline tables.
> > > >
> > > > Enis
> > > >
> > > >
> > > > On Thu, Mar 21, 2013 at 3:16 PM, Jean-Marc Spaggiari <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > >> Offline merge is already there and working fine.
> > > >>
> > > >> The usecase here was to retreive all the regions for a given table
> to
> > > >> merge them 2 by 2, offline.
> > > >>
> > > >> It's working fine, but since the Meta rework it's not working
> anymore
> > > >> and I'm trying to rebase the patch.
> > > >>
> > > >> Like J-D is saying, yes, it's used only in the offline merge... And
> > > >> since the online merge is coming, I think it's cleaner to keep the
> > > >> code in the offline merge since it will disapear soon, but in the
> > > >> meantime, at least, we will have the offline one.
> > > >>
> > > >> JM
> > > >>
> > > >> 2013/3/21 Enis Söztutar <[EMAIL PROTECTED]>:
> > > >> > What is the use case behind offline merge? Is it because we cannot
> > do
> > > >> > online merge yet? If we can get HBASE-7403 in, is there still need
> > to
> > > >> > support offline merge?
> > > >> >
> > > >> > Enis
> > > >> >
> > > >> >
> > > >> > On Thu, Mar 21, 2013 at 2:56 PM, Jean-Daniel Cryans <
> > > [EMAIL PROTECTED]
> > > >> >wrote:
> > > >> >
> > > >> >> As far as I can tell, only the merge code uses MetaUtils to do
> > > offline
> > > >> >> work. If this is the code you are in then pull it back into
> > MetaUtils
> > > >> >> I think.
> > > >> >>
> > > >> >> J-D
> > > >> >>
> > > >> >> On Thu, Mar 21, 2013 at 2:44 PM, Jean-Marc Spaggiari
> > > >> >> <[EMAIL PROTECTED]> wrote:
> > > >> >> > Vector is because of a very old bad habit ;) I will change that
> > to
> > > >> >> ArrayList.
> > > >> >> >
> > > >> >> > So far I have inlined the scanMetaRegion feature into the
> Merge,
> > > but
> > > >> >> > maybe it should be cleaner to put it back in?
> > > >> >> >
> > > >> >> > Anyway, I will keep the inlined one until everything is
> cleaned.
> > > >> >> >
> > > >> >> > 2013/3/21 Enis Söztutar <[EMAIL PROTECTED]>:
> > > >> >> >> The problem around current META scanning is that there is more
> > > than
> > > >> one