Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Should compaction be triggered even if User periodically flushes


+
ramkrishna vasudevan 2013-04-05, 09:00
+
Jean-Marc Spaggiari 2013-04-05, 11:20
+
ramkrishna vasudevan 2013-04-05, 16:50
+
Jean-Daniel Cryans 2013-04-05, 16:57
+
Sergey Shelukhin 2013-04-05, 17:25
+
ramkrishna vasudevan 2013-04-05, 17:44
Copy link to this message
-
Re: Should compaction be triggered even if User periodically flushes
Raised HBASE-8282 with log snippets.

Regards
Ram
On Fri, Apr 5, 2013 at 11:14 PM, ramkrishna vasudevan <
[EMAIL PROTECTED]> wrote:

> I will raise a JIRA and attach the logs.  I saw that in 0.94 also the
> behaviour remains the same.
> I did not change any of the default configruations.
>
> Regards
> Ram
>
>
> On Fri, Apr 5, 2013 at 10:55 PM, Sergey Shelukhin <[EMAIL PROTECTED]>wrote:
>
>> +1... logs would be interesting to see, unless there was configuration
>> difference it should be the same.
>> The only thing that comes to mind if off-peak hours starting maybe?
>>
>> On Fri, Apr 5, 2013 at 9:57 AM, Jean-Daniel Cryans <[EMAIL PROTECTED]
>> >wrote:
>>
>> > That sounds like a bug to me. Can you find why it's not being compacted?
>> > Was this behavior changed at some point?
>> >
>> > Thx,
>> >
>> > J-D
>> >
>> >
>> > On Fri, Apr 5, 2013 at 9:50 AM, ramkrishna vasudevan <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> > > Hi JM
>> > > Thanks for your reply.  My concern is not with respect to compaction
>> > > happening when a region is opening in another RS, my point was though
>> the
>> > > flushes were happening and more storefiles are created
>> > > by user initiated flushing, the compaction were not getting triggered.
>> > >
>> > >
>> > > On Fri, Apr 5, 2013 at 4:50 PM, Jean-Marc Spaggiari <
>> > > [EMAIL PROTECTED]
>> > > > wrote:
>> > >
>> > > > Hi Ram,
>> > > >
>> > > > What do you think is wrong with the approach below?
>> > > >
>> > > > Doing the compaction when the region is opened in another RS will
>> > > > improve data locality.
>> > > >
>> > > > JM
>> > > >
>> > > > 2013/4/5 ramkrishna vasudevan <[EMAIL PROTECTED]>:
>> > > > > Hi devs
>> > > > >
>> > > > > If user periodically flushes the memstore we keep creating Store
>> > files.
>> > > > >  Now if the number of store files  satisfies the compaction
>> policy we
>> > > > dont
>> > > > > do compaction.
>> > > > >
>> > > > > We print the msg saying compaction_requested=true but do not
>> include
>> > > into
>> > > > > the compactsplitthread.
>> > > > >
>> > > > > But when the RS goes down and the region gets opened in another
>> RS as
>> > > > part
>> > > > > of postOpenDeployTasks we do compaction because the store file
>> count
>> > > has
>> > > > > satisfied the compaction criteria.
>> > > > >
>> > > > > What could be the right approach here?
>> > > > >
>> > > > > Regards
>> > > > > Ram
>> > > >
>> > >
>> >
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB