Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> reginoserver's log analysis?


Copy link to this message
-
Re: reginoserver's log analysis?
HI,
     awfully appreciate to you all.
     i prefer the network bandwidth problem, cause it is easy to work.

     Thanks!
     beatls.

On Fri, Jan 4, 2013 at 2:32 AM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> It would delay this problem at least.  At the end of the day your
> application and use case is what determines what you should do. It seems to
> me you may be hot spotting this region.  Perhaps you should check your
> schema to see if there is something you can do to better distribute your
> writes.
>
> If you have the memory to spare, then yes increase the multiplier.
>  Otherwise look for ways to avoid hot spots, or heavy writes to a single
> region.
>
>
> On Thu, Jan 3, 2013 at 1:25 PM, hua beatls <[EMAIL PROTECTED]> wrote:
>
> > HI,
> >    I think you have this experience, and i plan to increase the network
> > bandwith.
> >    but   some one told me that increase the
> > 'hbase.hregion.memstore.block.multiplier" from default 2 to 4 could solve
> > this problem.
> >    which way to go?
> >
> >
> >    Thanks!
> >     bealts
> >
> >
> > On Fri, Jan 4, 2013 at 1:37 AM, Asaf Mesika <[EMAIL PROTECTED]>
> wrote:
> >
> > > When I encountered this error it was caused by slow network which
> caused
> > > hdfs to be slow which caused flush to take more time thus blocking
> > updates
> > > occurred.
> > >
> > > Sent from my iPhone
> > >
> > > On 3 בינו 2013, at 18:25, ramkrishna vasudevan <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > You need to see your application to see how much is your rate of puts.
> > > Also how many column families do you have?
> > > All these factors will help you in determining your memstore size.
> > >
> > > Regards
> > > Ram
> > >
> > > On Thu, Jan 3, 2013 at 9:51 PM, hua beatls <[EMAIL PROTECTED]> wrote:
> > >
> > > HI,
> > >
> > >   what i mean is shoud we increate the memstore according to this log?
> > >
> > >
> > >   Thanks!
> > >
> > >
> > >   beatls.
> > >
> > >
> > > On Thu, Jan 3, 2013 at 8:37 PM, Anoop Sam John <[EMAIL PROTECTED]>
> > wrote:
> > >
> > >
> > >
> > > I guess you mean  "what is blocking memstore size"
> > >
> > > This you can configure using 2 properties.
> > >
> > > 1. hbase.hregion.memstore.flush.size -> Using this you specify when the
> > >
> > > memstore to be flushed as a file. The default value is 128MB. So when
> the
> > >
> > > region memstore size reaches this value a flush will be initiated.
> > >
> > > 2. hbase.hregion.memstore.block.multiplier -> deafult value is 2
> > >
> > > The blocking memstore size is calculated as flushSize multiplied by
> this
> > >
> > > multiplier.  So the default value will be 256MB.
> > >
> > > What value to set depends on the total memory available for the process
> > >
> > > in
> > >
> > > the RegionServers and the total no# of regions per server.
> > >
> > > Say you set this blocking size as 512m and there can be 100 regions per
> > >
> > > region server in your cluster, then you are *possibly allowing* a total
> > >
> > > memory of 512*100 mb for the memstores alone. [RS process will not
> other
> > >
> > > memory also for its normal operation].  Take care of these aspects when
> > >
> > > you
> > >
> > > design
> > >
> > >
> > > -Anoop-
> > >
> > >
> > > ________________________________________
> > >
> > > From: hua beatls [[EMAIL PROTECTED]]
> > >
> > > Sent: Thursday, January 03, 2013 5:41 PM
> > >
> > > To: [EMAIL PROTECTED]
> > >
> > > Subject: Re: reginoserver's log analysis?
> > >
> > >
> > > HI,
> > >
> > >    what is the block size? how to set it ?  and shoud i increate it for
> > >
> > > the memstore ?
> > >
> > >
> > >    Thanks!
> > >
> > >     beatls
> > >
> > >
> > > On Thu, Jan 3, 2013 at 6:23 PM, Anoop Sam John <[EMAIL PROTECTED]>
> > >
> > > wrote:
> > >
> > >
> > > The writes to your table will be written 1st to in memory data
> > >
> > > structure
> > >
> > > (memstore). When this memstore reaches some defined size it will be
> > >
> > > flushed
> > >
> > > into the file system and the memory can be cleared.