Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Should compaction be triggered even if User periodically flushes


Copy link to this message
-
Re: Should compaction be triggered even if User periodically flushes
ramkrishna vasudevan 2013-04-05, 17:44
I will raise a JIRA and attach the logs.  I saw that in 0.94 also the
behaviour remains the same.
I did not change any of the default configruations.

Regards
Ram
On Fri, Apr 5, 2013 at 10:55 PM, Sergey Shelukhin <[EMAIL PROTECTED]>wrote:

> +1... logs would be interesting to see, unless there was configuration
> difference it should be the same.
> The only thing that comes to mind if off-peak hours starting maybe?
>
> On Fri, Apr 5, 2013 at 9:57 AM, Jean-Daniel Cryans <[EMAIL PROTECTED]
> >wrote:
>
> > That sounds like a bug to me. Can you find why it's not being compacted?
> > Was this behavior changed at some point?
> >
> > Thx,
> >
> > J-D
> >
> >
> > On Fri, Apr 5, 2013 at 9:50 AM, ramkrishna vasudevan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi JM
> > > Thanks for your reply.  My concern is not with respect to compaction
> > > happening when a region is opening in another RS, my point was though
> the
> > > flushes were happening and more storefiles are created
> > > by user initiated flushing, the compaction were not getting triggered.
> > >
> > >
> > > On Fri, Apr 5, 2013 at 4:50 PM, Jean-Marc Spaggiari <
> > > [EMAIL PROTECTED]
> > > > wrote:
> > >
> > > > Hi Ram,
> > > >
> > > > What do you think is wrong with the approach below?
> > > >
> > > > Doing the compaction when the region is opened in another RS will
> > > > improve data locality.
> > > >
> > > > JM
> > > >
> > > > 2013/4/5 ramkrishna vasudevan <[EMAIL PROTECTED]>:
> > > > > Hi devs
> > > > >
> > > > > If user periodically flushes the memstore we keep creating Store
> > files.
> > > > >  Now if the number of store files  satisfies the compaction policy
> we
> > > > dont
> > > > > do compaction.
> > > > >
> > > > > We print the msg saying compaction_requested=true but do not
> include
> > > into
> > > > > the compactsplitthread.
> > > > >
> > > > > But when the RS goes down and the region gets opened in another RS
> as
> > > > part
> > > > > of postOpenDeployTasks we do compaction because the store file
> count
> > > has
> > > > > satisfied the compaction criteria.
> > > > >
> > > > > What could be the right approach here?
> > > > >
> > > > > Regards
> > > > > Ram
> > > >
> > >
> >
>