Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> reginoserver's log analysis?


+
hua beatls 2013-01-03, 09:45
+
Anoop Sam John 2013-01-03, 10:23
+
hua beatls 2013-01-03, 12:11
+
Anoop Sam John 2013-01-03, 12:37
+
hua beatls 2013-01-03, 16:21
+
ramkrishna vasudevan 2013-01-03, 16:24
+
Asaf Mesika 2013-01-03, 17:37
+
hua beatls 2013-01-03, 18:25
+
Bryan Beaudreault 2013-01-03, 18:32
Copy link to this message
-
Re: reginoserver's log analysis?
HI,
     awfully appreciate to you all.
     i prefer the network bandwidth problem, cause it is easy to work.

     Thanks!
     beatls.

On Fri, Jan 4, 2013 at 2:32 AM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> It would delay this problem at least.  At the end of the day your
> application and use case is what determines what you should do. It seems to
> me you may be hot spotting this region.  Perhaps you should check your
> schema to see if there is something you can do to better distribute your
> writes.
>
> If you have the memory to spare, then yes increase the multiplier.
>  Otherwise look for ways to avoid hot spots, or heavy writes to a single
> region.
>
>
> On Thu, Jan 3, 2013 at 1:25 PM, hua beatls <[EMAIL PROTECTED]> wrote:
>
> > HI,
> >    I think you have this experience, and i plan to increase the network
> > bandwith.
> >    but   some one told me that increase the
> > 'hbase.hregion.memstore.block.multiplier" from default 2 to 4 could solve
> > this problem.
> >    which way to go?
> >
> >
> >    Thanks!
> >     bealts
> >
> >
> > On Fri, Jan 4, 2013 at 1:37 AM, Asaf Mesika <[EMAIL PROTECTED]>
> wrote:
> >
> > > When I encountered this error it was caused by slow network which
> caused
> > > hdfs to be slow which caused flush to take more time thus blocking
> > updates
> > > occurred.
> > >
> > > Sent from my iPhone
> > >
> > > On 3 בינו 2013, at 18:25, ramkrishna vasudevan <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > You need to see your application to see how much is your rate of puts.
> > > Also how many column families do you have?
> > > All these factors will help you in determining your memstore size.
> > >
> > > Regards
> > > Ram
> > >
> > > On Thu, Jan 3, 2013 at 9:51 PM, hua beatls <[EMAIL PROTECTED]> wrote:
> > >
> > > HI,
> > >
> > >   what i mean is shoud we increate the memstore according to this log?
> > >
> > >
> > >   Thanks!
> > >
> > >
> > >   beatls.
> > >
> > >
> > > On Thu, Jan 3, 2013 at 8:37 PM, Anoop Sam John <[EMAIL PROTECTED]>
> > wrote:
> > >
> > >
> > >
> > > I guess you mean  "what is blocking memstore size"
> > >
> > > This you can configure using 2 properties.
> > >
> > > 1. hbase.hregion.memstore.flush.size -> Using this you specify when the
> > >
> > > memstore to be flushed as a file. The default value is 128MB. So when
> the
> > >
> > > region memstore size reaches this value a flush will be initiated.
> > >
> > > 2. hbase.hregion.memstore.block.multiplier -> deafult value is 2
> > >
> > > The blocking memstore size is calculated as flushSize multiplied by
> this
> > >
> > > multiplier.  So the default value will be 256MB.
> > >
> > > What value to set depends on the total memory available for the process
> > >
> > > in
> > >
> > > the RegionServers and the total no# of regions per server.
> > >
> > > Say you set this blocking size as 512m and there can be 100 regions per
> > >
> > > region server in your cluster, then you are *possibly allowing* a total
> > >
> > > memory of 512*100 mb for the memstores alone. [RS process will not
> other
> > >
> > > memory also for its normal operation].  Take care of these aspects when
> > >
> > > you
> > >
> > > design
> > >
> > >
> > > -Anoop-
> > >
> > >
> > > ________________________________________
> > >
> > > From: hua beatls [[EMAIL PROTECTED]]
> > >
> > > Sent: Thursday, January 03, 2013 5:41 PM
> > >
> > > To: [EMAIL PROTECTED]
> > >
> > > Subject: Re: reginoserver's log analysis?
> > >
> > >
> > > HI,
> > >
> > >    what is the block size? how to set it ?  and shoud i increate it for
> > >
> > > the memstore ?
> > >
> > >
> > >    Thanks!
> > >
> > >     beatls
> > >
> > >
> > > On Thu, Jan 3, 2013 at 6:23 PM, Anoop Sam John <[EMAIL PROTECTED]>
> > >
> > > wrote:
> > >
> > >
> > > The writes to your table will be written 1st to in memory data
> > >
> > > structure
> > >
> > > (memstore). When this memstore reaches some defined size it will be
> > >
> > > flushed
> > >
> > > into the file system and the memory can be cleared.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB