Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Default value for Periodic Flusher


Copy link to this message
-
Re: Default value for Periodic Flusher
Should we default it to 0 (off) then? (As Ted pointed out, you can turn this off).
Are you not worried about MTTR when a RegionServer dies with 3GB of logs to replay?
-- Lars

________________________________
 From: Adrien Mogenet <[EMAIL PROTECTED]>
To: user <[EMAIL PROTECTED]>
Sent: Tuesday, December 10, 2013 2:16 PM
Subject: Re: Default value for Periodic Flusher
 

Hi guys,

I've upgraded to 0.94.11. Here is my "worst-case scenario" :

- let say each regionserver has 3 GB memstore
- let say compaction max filesize is ~200 GB, min. 2 files, max 10 files.
- let say memstore is growing "slowly" (1 GB / hour per RS)

Then, automatically flushing every hour will lead into 1 GB storefiles,
being compacted into storefiles of 2 GB, 3 GB, 4.... up to 200 GB.
Sometimes, my write-load becomes very low, and periodic flusher will flush
perhaps 1 MB of data, it will trigger a minor compaction of hundreds
gigabytes + 1 MB; it seems to be lots of IO just to merge 1 MB of data.

Previously (ie. lack of periodic flusher) memstore was creating 3 GB
storefiles, and thus creating (after minor compactions) 3 GB, 6 GB, 9 GB...
up to 200 GB storefiles. And if memstore is growing slowly, it won't
generate small storefiles on HDFS. If think it looks like a more reasonable
IO-load, doesn't it?

I deeply agree with Periodic Flusher relevance, but I don't think it's
suitable for everyone. Do you share my opinion wrt. my workload?
On Sun, Dec 8, 2013 at 10:36 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Adrien:
> This config was introduced in 0.94.8
>
> Which release did you upgrade to ?
>
> As Jean-Marc said, telling us the issue (along with log snippet) would
> help.
>
> Cheers
>
>
> On Mon, Dec 9, 2013 at 1:26 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]
> > wrote:
>
> > Hi Adrien,
> >
> > What was the bad behavior you faced because of that? Maybe it's what need
> > to be fixed more than the periodic flusher? Or put a bigger default
> value?
> >
> > JM
> >
> >
> > 2013/12/8 Adrien Mogenet <[EMAIL PROTECTED]>
> >
> > > Hi there,
> > >
> > > I'm wondering if the Periodic Flusher should be disabled by default?
> > >
> > > During a recent upgrade, I've noticed a strange behavior modification
> on
> > my
> > > servers, and it was due to this new feature, supplied with a "1 hour"
> > > default value. I think upgrades should keep default behavior as close
> as
> > in
> > > previous versions. "By chance", logs were mentioning this "Periodic
> > > Flusher" but this could have taken much more time to debug otherwise
> :-)
> > >
> > > What are you thoughts guys?
> > > (Perhaps should be cc'ed to dev list?)
> > > --
> > > Adrien Mogenet
> > > http://www.borntosegfault.com

> > >
> >
>

--
Adrien Mogenet
http://www.borntosegfault.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB