Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Regionserver goes down while endpoint execution


Copy link to this message
-
Re: Regionserver goes down while endpoint execution
I dont have any sample code to share with you.  Instead of calling
HTable#put() from the postPut() impl, add these Put objects (Index table
Put) to some Q and the CP flow immediately comes out.(Not waiting for the
write on index table) Some dedicated pool of worker threads can poll the
index puts from Q and do the actual put() operation with index table. As
Adrien said this will be eventual consistent model and what if before the
worker thread actually doing the put on index table, the RS is going down.
So these consistency related points to be taken care. Hope I made it clear
to you :)

-Anoop-

On Tue, Mar 26, 2013 at 1:57 PM, Kumar, Deepak8 <[EMAIL PROTECTED]>wrote:

> Hi Adrien,
> From CP hook, do we mean EndPoints implementation? Do we have any document
> for asynchronous put implementations? A simple example would also be quite
> better to start with.
>
> Regards,
> Deepak
>
> -----Original Message-----
> From: Adrien Mogenet [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, March 26, 2013 3:42 AM
> To: [EMAIL PROTECTED]
> Cc: Anoop Sam John; [EMAIL PROTECTED]
> Subject: Re: Regionserver goes down while endpoint execution
>
> Sure, and it might be costly if your CP is building secondary index through
> `Put' to external tables/regions. Each of your RPC call will generate
> another RPC call. Also, take note that your client could get an unexpected
> behavior; it might be blocked waiting for RPC answer just because of your
> CP.
>
> As Anoop said, rethink on doing such things from your CP hook. Otherwise,
> think about implementing asynchronous `Put' using a thread but it might
> lead in some headach (ensure - eventually - consistency, handle
> reconections, etc.)
>
>
> On Tue, Mar 26, 2013 at 8:27 AM, Kumar, Deepak8 <[EMAIL PROTECTED]
> >wrote:
>
> > Thanks Anoop for assisting.
> >
> > I don't have Put(List). All those calls are independent of each other.
> Let
> > me put it this way. Suppose I have 1000 insertions/seconds on a table,
> > (each insert is independent & not coming in a list). Would it be a
> > performance impact to use CP to update secondary index?
> >
> > Regards,
> > Deepak
> >
> > -----Original Message-----
> > From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> > Sent: Tuesday, March 26, 2013 2:20 AM
> > To: Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; '
> > [EMAIL PROTECTED]'
> > Subject: RE: Regionserver goes down while endpoint execution
> >
> > Deepak
> >      When you say 1000 invocations, you mean you are having a put(List)
> > with 1000 items?  In the region level when it deals with a batch of Puts,
> > it calls postPut() for each of the Put one after the other. In the same
> > handler thread which is doing the write. So yes one need to wait for the
> > other. Otherwise you need to handle within the implementation of having a
> > thread pool or so.
> >
> > I would say rethink on doing HTable#put() from a CP hook.
> >
> > -Anoop-
> >
> > ________________________________________
> > From: Kumar, Deepak8  [[EMAIL PROTECTED]]
> > Sent: Monday, March 25, 2013 10:23 PM
> > To: '[EMAIL PROTECTED]'; '[EMAIL PROTECTED]'; Anoop
> > Sam John
> > Subject: RE: Regionserver goes down while endpoint execution
> >
> > Hi Anoop,
> >
> > Could you please provide some pointers. If we have a regionbase
> > coprocessor which has some set of operation implemented in its postPut
> > implementation.
> >
> > If I have say 1000 invocations of postPut per seconds, are those calls
> > would be blocking? I mean, the second call will only start  after the
> > completion of first call of postPut, or it would execute independently?
> >
> > Regards,
> > Deepak
> > -----Original Message-----
> > From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> > Sent: Wednesday, March 20, 2013 8:59 AM
> > To: Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; '
> > [EMAIL PROTECTED]'
> > Subject: RE: Regionserver goes down while endpoint execution
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB