Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Does HBase combine multiple Puts against the same region server?


Copy link to this message
-
Re: Does HBase combine multiple Puts against the same region server?
The code does the the following:

1) Add put to buffer
2) If buffer is > size (2MB) - then flush the puts
3) Finally, for the left over puts in the buffer - if auto flush if on
(which is the default), flush the puts

When you flush your puts, the puts are grouped by regionserver and a single
call is made to each region server - so this could be 1 call or n calls if
you have n region servers. If you want to flush a large payload - you can
consider increasing the buffer size.

On Thu, Dec 6, 2012 at 11:17 AM, Jimmy Xiang <[EMAIL PROTECTED]> wrote:

> This has been built in hbase for quite some time, no application change.
>
> On Thu, Dec 6, 2012 at 11:08 AM, yun peng <[EMAIL PROTECTED]> wrote:
> > Is that done in current HBase implementation (say, 0.94.2 or more recent
> > one) or it requires applications to handle it?
> > Thanks for your note,
> > Yun
> >
> > On Thu, Dec 6, 2012 at 1:42 PM, Jimmy Xiang <[EMAIL PROTECTED]> wrote:
> >
> >> If auto flush is off, multiple puts could be combined into a batch and
> >> send to the region server in one RPC call if they are for the same
> >> region server.
> >>
> >> Thanks,
> >> Jimmy
> >>
> >> On Thu, Dec 6, 2012 at 10:34 AM, yun peng <[EMAIL PROTECTED]>
> wrote:
> >> > Hi, I have question on how the multiple Puts() are executed when they
> are
> >> > issued against the same region server.
> >> >
> >> > For example, in the case of asynchronous executing Put() using
> >> > setAutoFlush(true), there will be multiple Puts() in the writeBuffer.
> Or
> >> > use HTbale API put(List puts) which directly issues multiple Puts. In
> >> > either case, let's say, would two Puts in the list, which are issued
> >> > against the same HRegionServer, be combined in a single RPC before
> >> sending
> >> > to that RegionServer? I appreciate it if I can get pointer to the
> code in
> >> > HBase. Thanks...
> >> >
> >> > Regards,
> >> > Yun
> >>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB