Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Regionserver goes down while endpoint execution


Copy link to this message
-
Re: Regionserver goes down while endpoint execution
ramkrishna vasudevan 2013-03-28, 10:53
Yes, you may have to write  your custom load balancer that does that.

REgards
Ram

On Thu, Mar 28, 2013 at 4:20 PM, Kumar, Deepak8 <[EMAIL PROTECTED]>wrote:

> Hi Anoop,
> Thanks for assisting. As Adrien said, it would be costly if we put to
> external tables/regions. Do we have a way so that we can force that the
> main table & secondary table is being loaded on same region server always,
> so that the Put is not as costly as in case of when they are distributed
> over network on different region servers?
>
> Regards,
> Deepak
>
> -----Original Message-----
> From: Anoop John [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, March 26, 2013 1:18 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Regionserver goes down while endpoint execution
>
> I dont have any sample code to share with you.  Instead of calling
> HTable#put() from the postPut() impl, add these Put objects (Index table
> Put) to some Q and the CP flow immediately comes out.(Not waiting for the
> write on index table) Some dedicated pool of worker threads can poll the
> index puts from Q and do the actual put() operation with index table. As
> Adrien said this will be eventual consistent model and what if before the
> worker thread actually doing the put on index table, the RS is going down.
> So these consistency related points to be taken care. Hope I made it clear
> to you :)
>
> -Anoop-
>
> On Tue, Mar 26, 2013 at 1:57 PM, Kumar, Deepak8 <[EMAIL PROTECTED]
> >wrote:
>
> > Hi Adrien,
> > From CP hook, do we mean EndPoints implementation? Do we have any
> document
> > for asynchronous put implementations? A simple example would also be
> quite
> > better to start with.
> >
> > Regards,
> > Deepak
> >
> > -----Original Message-----
> > From: Adrien Mogenet [mailto:[EMAIL PROTECTED]]
> > Sent: Tuesday, March 26, 2013 3:42 AM
> > To: [EMAIL PROTECTED]
> > Cc: Anoop Sam John; [EMAIL PROTECTED]
> > Subject: Re: Regionserver goes down while endpoint execution
> >
> > Sure, and it might be costly if your CP is building secondary index
> through
> > `Put' to external tables/regions. Each of your RPC call will generate
> > another RPC call. Also, take note that your client could get an
> unexpected
> > behavior; it might be blocked waiting for RPC answer just because of your
> > CP.
> >
> > As Anoop said, rethink on doing such things from your CP hook. Otherwise,
> > think about implementing asynchronous `Put' using a thread but it might
> > lead in some headach (ensure - eventually - consistency, handle
> > reconections, etc.)
> >
> >
> > On Tue, Mar 26, 2013 at 8:27 AM, Kumar, Deepak8 <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Thanks Anoop for assisting.
> > >
> > > I don't have Put(List). All those calls are independent of each other.
> > Let
> > > me put it this way. Suppose I have 1000 insertions/seconds on a table,
> > > (each insert is independent & not coming in a list). Would it be a
> > > performance impact to use CP to update secondary index?
> > >
> > > Regards,
> > > Deepak
> > >
> > > -----Original Message-----
> > > From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> > > Sent: Tuesday, March 26, 2013 2:20 AM
> > > To: Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; '
> > > [EMAIL PROTECTED]'
> > > Subject: RE: Regionserver goes down while endpoint execution
> > >
> > > Deepak
> > >      When you say 1000 invocations, you mean you are having a put(List)
> > > with 1000 items?  In the region level when it deals with a batch of
> Puts,
> > > it calls postPut() for each of the Put one after the other. In the same
> > > handler thread which is doing the write. So yes one need to wait for
> the
> > > other. Otherwise you need to handle within the implementation of
> having a
> > > thread pool or so.
> > >
> > > I would say rethink on doing HTable#put() from a CP hook.
> > >
> > > -Anoop-
> > >
> > > ________________________________________
> > > From: Kumar, Deepak8  [[EMAIL PROTECTED]]
> > > Sent: Monday, March 25, 2013 10:23 PM