Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Regionserver goes down while endpoint execution


+
Kumar, Deepak8 2013-03-12, 05:51
+
lars hofhansl 2013-03-12, 06:01
+
Kumar, Deepak8 2013-03-12, 06:27
+
Kumar, Deepak8 2013-03-12, 06:59
+
Kumar, Deepak8 2013-03-12, 11:46
+
Ted Yu 2013-03-12, 16:29
+
Gary Helmling 2013-03-12, 18:13
+
Kumar, Deepak8 2013-03-13, 15:19
+
Ted Yu 2013-03-13, 16:01
+
Himanshu Vashishtha 2013-03-13, 16:08
+
Kumar, Deepak8 2013-03-14, 17:09
+
Ted Yu 2013-03-14, 17:15
+
Himanshu Vashishtha 2013-03-14, 17:45
+
Anoop Sam John 2013-03-15, 06:55
+
Kumar, Deepak8 2013-03-20, 07:41
+
ramkrishna vasudevan 2013-03-20, 08:01
+
Anoop Sam John 2013-03-20, 08:36
+
Kumar, Deepak8 2013-03-20, 12:44
+
Anoop Sam John 2013-03-20, 12:58
+
Kumar, Deepak8 2013-03-20, 13:18
+
Kumar, Deepak8 2013-03-25, 16:53
Copy link to this message
-
RE: Regionserver goes down while endpoint execution
Anoop Sam John 2013-03-26, 06:20
Deepak
     When you say 1000 invocations, you mean you are having a put(List) with 1000 items?  In the region level when it deals with a batch of Puts, it calls postPut() for each of the Put one after the other. In the same handler thread which is doing the write. So yes one need to wait for the other. Otherwise you need to handle within the implementation of having a thread pool or so.

I would say rethink on doing HTable#put() from a CP hook.

-Anoop-

________________________________________
From: Kumar, Deepak8  [[EMAIL PROTECTED]]
Sent: Monday, March 25, 2013 10:23 PM
To: '[EMAIL PROTECTED]'; '[EMAIL PROTECTED]'; Anoop Sam John
Subject: RE: Regionserver goes down while endpoint execution

Hi Anoop,

Could you please provide some pointers. If we have a regionbase coprocessor which has some set of operation implemented in its postPut implementation.

If I have say 1000 invocations of postPut per seconds, are those calls would be blocking? I mean, the second call will only start  after the completion of first call of postPut, or it would execute independently?

Regards,
Deepak
-----Original Message-----
From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, March 20, 2013 8:59 AM
To: Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; '[EMAIL PROTECTED]'
Subject: RE: Regionserver goes down while endpoint execution

What is this doing?

HbaseTableAccess.getInstance().connectTable(tableName);

Can you take a look at HTablePool?

-Anoop-

________________________________
From: Kumar, Deepak8 [[EMAIL PROTECTED]]
Sent: Wednesday, March 20, 2013 6:14 PM
To: '[EMAIL PROTECTED]'; Anoop Sam John; '[EMAIL PROTECTED]'
Subject: RE: Regionserver goes down while endpoint execution
Hi,

Even when I moved table = HbaseTableAccess.getInstance().connectTable(tableName); to postPut method of coprocessor, I am getting the same exception.

Regards,

Deepak

-----Original Message-----
From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, March 20, 2013 4:37 AM
To: [EMAIL PROTECTED]
Subject: RE: Regionserver goes down while endpoint execution

Hi Deepak

                      Yes Ram told it correctly regarding the thread safety issue with HTable.

I can see you issue HTable#put() to write to index table and this call you are making from a CP.

Our design was around making the actual table region and corresponding index table region collocated in the same RS. In our impl we are not calling HTable#put() from CP. We get the index region and directly call put on that.

In some mail thread Andrew Purtell explained it nicely that doing a client based call like this from CP is an anti pattern. Think of avoiding this if you are going with CP based impl.  If you are doing with the current way or you need a global ordered indexing, IMHO think of managing the index from the client side only rather than CP.

-Anoop-

________________________________________

From: ramkrishna vasudevan [[EMAIL PROTECTED]]

Sent: Wednesday, March 20, 2013 1:31 PM

To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>

Subject: Re: Regionserver goes down while endpoint execution

Hi Deepak

"this.table.put(putInIndexTable);""

I think this one should be the problem.

Your table is at the instance level.   See the documentation of HTable.

"<p>This class is not thread safe for reads nor write."

So if you try creating a new HTable every time this problem should be

avoided.

Regards

Ram

On Wed, Mar 20, 2013 at 1:11 PM, Kumar, Deepak8 <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>wrote:

> Hi Anoop,

>

> Quite inspired by your coprocessor secondary indexing document & trying to

> implement one, for better response :)

>

>

>

> The coprocessor executes for some time, but later on (say after 400-500

> inserts) it gives IndexOutofBounds exception.

>

>

>

> The stack trace is

>

> 2013-03-20 02:40:42,074 INFO

> com.citi.sponge.hbase.coprocessor.secondaryindex.SecondaryIndexCoprocessor:























































































































































































































+
Kumar, Deepak8 2013-03-26, 07:27
+
Adrien Mogenet 2013-03-26, 07:42
+
Kumar, Deepak8 2013-03-26, 08:27
+
Anoop John 2013-03-26, 17:17
+
Kumar, Deepak8 2013-03-28, 10:50
+
ramkrishna vasudevan 2013-03-28, 10:53
+
Agarwal, Saurabh 2013-03-28, 12:26
+
Anoop Sam John 2013-04-02, 06:51
+
Kumar, Deepak8 2013-03-28, 12:11
+
Himanshu Vashishtha 2013-03-12, 16:59