Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Regionserver goes down while endpoint execution


Copy link to this message
-
Re: Regionserver goes down while endpoint execution
Adrien Mogenet 2013-03-26, 07:42
Sure, and it might be costly if your CP is building secondary index through
`Put' to external tables/regions. Each of your RPC call will generate
another RPC call. Also, take note that your client could get an unexpected
behavior; it might be blocked waiting for RPC answer just because of your
CP.

As Anoop said, rethink on doing such things from your CP hook. Otherwise,
think about implementing asynchronous `Put' using a thread but it might
lead in some headach (ensure - eventually - consistency, handle
reconections, etc.)
On Tue, Mar 26, 2013 at 8:27 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>wrote:

> Thanks Anoop for assisting.
>
> I don't have Put(List). All those calls are independent of each other. Let
> me put it this way. Suppose I have 1000 insertions/seconds on a table,
> (each insert is independent & not coming in a list). Would it be a
> performance impact to use CP to update secondary index?
>
> Regards,
> Deepak
>
> -----Original Message-----
> From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, March 26, 2013 2:20 AM
> To: Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; '
> [EMAIL PROTECTED]'
> Subject: RE: Regionserver goes down while endpoint execution
>
> Deepak
>      When you say 1000 invocations, you mean you are having a put(List)
> with 1000 items?  In the region level when it deals with a batch of Puts,
> it calls postPut() for each of the Put one after the other. In the same
> handler thread which is doing the write. So yes one need to wait for the
> other. Otherwise you need to handle within the implementation of having a
> thread pool or so.
>
> I would say rethink on doing HTable#put() from a CP hook.
>
> -Anoop-
>
> ________________________________________
> From: Kumar, Deepak8  [[EMAIL PROTECTED]]
> Sent: Monday, March 25, 2013 10:23 PM
> To: '[EMAIL PROTECTED]'; '[EMAIL PROTECTED]'; Anoop
> Sam John
> Subject: RE: Regionserver goes down while endpoint execution
>
> Hi Anoop,
>
> Could you please provide some pointers. If we have a regionbase
> coprocessor which has some set of operation implemented in its postPut
> implementation.
>
> If I have say 1000 invocations of postPut per seconds, are those calls
> would be blocking? I mean, the second call will only start  after the
> completion of first call of postPut, or it would execute independently?
>
> Regards,
> Deepak
> -----Original Message-----
> From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, March 20, 2013 8:59 AM
> To: Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; '
> [EMAIL PROTECTED]'
> Subject: RE: Regionserver goes down while endpoint execution
>
> What is this doing?
>
> HbaseTableAccess.getInstance().connectTable(tableName);
>
>
>
> Can you take a look at HTablePool?
>
>
>
> -Anoop-
>
>
>
> ________________________________
> From: Kumar, Deepak8 [[EMAIL PROTECTED]]
> Sent: Wednesday, March 20, 2013 6:14 PM
> To: '[EMAIL PROTECTED]'; Anoop Sam John; '
> [EMAIL PROTECTED]'
> Subject: RE: Regionserver goes down while endpoint execution
>
>
> Hi,
>
> Even when I moved table > HbaseTableAccess.getInstance().connectTable(tableName); to postPut method
> of coprocessor, I am getting the same exception.
>
>
>
> Regards,
>
> Deepak
>
>
>
> -----Original Message-----
> From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, March 20, 2013 4:37 AM
> To: [EMAIL PROTECTED]
> Subject: RE: Regionserver goes down while endpoint execution
>
>
>
> Hi Deepak
>
>                       Yes Ram told it correctly regarding the thread
> safety issue with HTable.
>
> I can see you issue HTable#put() to write to index table and this call you
> are making from a CP.
>
> Our design was around making the actual table region and corresponding
> index table region collocated in the same RS. In our impl we are not
> calling HTable#put() from CP. We get the index region and directly call put
> on that.
>
>
>
> In some mail thread Andrew Purtell explained it nicely that doing a client

Adrien Mogenet
06.59.16.64.22
http://www.mogenet.me