-Re: timeouts with lots of coprocessor puts on single row
Asaf Mesika 2013-08-26, 20:28
We did the same but on the client side, without any issue
On Monday, August 26, 2013, Olle Mårtensson wrote:
> I have developed a coprocessor that is extending BaseRegionObserver and
> implements the
> postPut method. The postPut method scans the columns of the row that the
> put was issued on and calculates an aggregated based on these values, when
> this is done a row in another table is updated with the aggregated value.
> This works out fine until I put some stress on one row, then the threads on
> the regionserver hosting the table will freeze on flushing the put on the
> aggregated value.
> The client application basically do 100 concurrent puts on one row in a
> tight loop( on the table where the coprocessor is activated ).
> After that the client sleeps for a while and tries to fetch the aggregated
> value and here the client freezes and periodically burps out exceptions.
> It works if I don't run so many put's in parallel.
> The HBASE environment is pseudo distributed 0.94.11 with one regionserver.
> I have tried using a connection pool in the coprocessor, bumped up the
> heapsize of the regionServer and also to up the number of RPC threads for
> the regionserver but without luck.
> The pseudo code postPut would be something like this:
> vals = env.getRegion().get(get).getFamilyMap().values()
> agg_val = aggregate(vals)
> agg_table = env.getTable("aggregates")
> put = new Put()
> And the real clojure variant is:
> The hbase-site.xml:
> The regionserver stacktrace:
> The client exceptions:
> Thanks // Olle