Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> flushing + compactions after config change

Copy link to this message
Re: 答复: flushing + compactions after config change
On Fri, Jun 28, 2013 at 9:31 AM, Jean-Daniel Cryans <[EMAIL PROTECTED]>wrote:

> On Thu, Jun 27, 2013 at 4:27 PM, Viral Bajaria <[EMAIL PROTECTED]>
> wrote:
> It's not random, it picks the region with the most data in its memstores.

That's weird, because I see some of my regions which receive the least
amount of data in a given time period flushing before the regions that are
receiving data continuously. The reason I know this is because of the write
pattern. Some of my tables are in catch-up mode i.e. I am ingesting data
from the past and they always have something to do. While some tables are
not in catch-up mode and are just sitting idle for most of the time. Yet I
see high number of flushes for those regions too.
> I doubt that it's the fact that it's a major compaction that it's
> making everything worse. When a minor gets promoted into a major it's
> because we're already going to compact all the files, so we might as
> well get rid of some deletes at the same time. They are all getting
> selected because the files are within the selection ratio. I would not
> focus on this to resolve your problem.

I meant worse for my writes not for HBase as a whole.
> I haven't been closely following this thread, but have you posted a
> log snippet somewhere? It's usually much more telling and we eliminate
> a few levels of interpretation. Make sure it's at DEBUG, and that you
> grab a few hours of activity. Get the GC log for the same time as
> well. Drop this on a web server or pastebin if it fits.

The only log snippet that I posted was the flushing action. Also that log
was not everything, I had grep'd a few lines out. Let me collect some more
stats here and post it again. I just enabled GC logging on this server,
deployed the wrong config out initially which had no GC logging. I am not
sure how GC logs will help here given that I am at less than 50% heap space
used and so I would doubt a stop the world GC happening. Are you trying to
look for some other information ?
> Thx,
> J-D
Thanks to everyone on this list for their time!