Hi,

I would like to feed a servlet that is serving a long polling request with
notifications when certain rows in an hbase table change.

It seems that the best mechanism to achieve this would be to add a custom
RegionObserver. What I'm unsure about though is how to best issue the
callback. Does hbase provide an RPC mechanism that I could re-use or should
I consider this a separate problem entirely? Are there alternative ways to
get notified of row changes that better handle this problem?

Thanks!
Jan

Please be a little more specific about where the notifications should be
received.

Your custom RegionObserver lives on region server.

I guess you want notifications to be delivered to your client, outside the
cluster.

Cheers

On Tue, Feb 5, 2013 at 12:18 PM, Jan Althaus <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I would like to feed a servlet that is serving a long polling request with
> notifications when certain rows in an hbase table change.
>
> It seems that the best mechanism to achieve this would be to add a custom
> RegionObserver. What I'm unsure about though is how to best issue the
> callback. Does hbase provide an RPC mechanism that I could re-use or should
> I consider this a separate problem entirely? Are there alternative ways to
> get notified of row changes that better handle this problem?
>
> Thanks!
> Jan
>

Sorry for breaking the thread. It seems my subscription wasn't active at
the time the reply was posted.

The client receiving the notifications would be a separate process,
potentially on a different machine. It could obviously connect to the HBase
cluster though.

I can certainly see how delivering such notifications would be outside the
scope of HBase, but I wanted to make sure that I wasn't reinventing the
wheel by using a completely different RPC mechanism between the region
server and the servlet.

> Please be a little more specific about where the notifications should be
> received.
>
> Your custom RegionObserver lives on region server.
>
> I guess you want notifications to be delivered to your client, outside the
> cluster.
>
> Cheers
>
> On Tue, Feb 5, 2013 at 12:18 PM, Jan Althaus <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I would like to feed a servlet that is serving a long polling request
with
>> notifications when certain rows in an hbase table change.
>>
>> It seems that the best mechanism to achieve this would be to add a custom
>> RegionObserver. What I'm unsure about though is how to best issue the
>> callback. Does hbase provide an RPC mechanism that I could re-use or
should
>> I consider this a separate problem entirely? Are there alternative ways
to
>> get notified of row changes that better handle this problem?
>>
>> Thanks!
>> Jan
>>

You are better off using your own RPC. Thrift would be a good choice.

On Tuesday, February 5, 2013, Jan Althaus wrote:

> Sorry for breaking the thread. It seems my subscription wasn't active at
> the time the reply was posted.
>
> The client receiving the notifications would be a separate process,
> potentially on a different machine. It could obviously connect to the HBase
> cluster though.
>
> I can certainly see how delivering such notifications would be outside the
> scope of HBase, but I wanted to make sure that I wasn't reinventing the
> wheel by using a completely different RPC mechanism between the region
> server and the servlet.
>
> > Please be a little more specific about where the notifications should be
> > received.
> >
> > Your custom RegionObserver lives on region server.
> >
> > I guess you want notifications to be delivered to your client, outside
> the
> > cluster.
> >
> > Cheers
> >
> > On Tue, Feb 5, 2013 at 12:18 PM, Jan Althaus <[EMAIL PROTECTED]<javascript:;>>
> wrote:
> >
> >> Hi,
> >>
> >> I would like to feed a servlet that is serving a long polling request
> with
> >> notifications when certain rows in an hbase table change.
> >>
> >> It seems that the best mechanism to achieve this would be to add a
> custom
> >> RegionObserver. What I'm unsure about though is how to best issue the
> >> callback. Does hbase provide an RPC mechanism that I could re-use or
> should
> >> I consider this a separate problem entirely? Are there alternative ways
> to
> >> get notified of row changes that better handle this problem?
> >>
> >> Thanks!
> >> Jan
> >>
>
--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

I'd say add some sort of durable queue to receive the alert if you don't want to miss a message.

Essentially use a region server observer and write the alert to your favorite pub/sub solution.
The key then is starting the pub/sub queue, first so that your region server can connect to it.

If you want to get fancy, there's a bit of overkill that you could do...  like figure out a way to write the output to HDFS if that made sense.

Just some thoughts before I'm really awake...

-Mike

On Feb 5, 2013, at 10:45 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> You are better off using your own RPC. Thrift would be a good choice.
>
> On Tuesday, February 5, 2013, Jan Althaus wrote:
>
>> Sorry for breaking the thread. It seems my subscription wasn't active at
>> the time the reply was posted.
>>
>> The client receiving the notifications would be a separate process,
>> potentially on a different machine. It could obviously connect to the HBase
>> cluster though.
>>
>> I can certainly see how delivering such notifications would be outside the
>> scope of HBase, but I wanted to make sure that I wasn't reinventing the
>> wheel by using a completely different RPC mechanism between the region
>> server and the servlet.
>>
>>> Please be a little more specific about where the notifications should be
>>> received.
>>>
>>> Your custom RegionObserver lives on region server.
>>>
>>> I guess you want notifications to be delivered to your client, outside
>> the
>>> cluster.
>>>
>>> Cheers
>>>
>>> On Tue, Feb 5, 2013 at 12:18 PM, Jan Althaus <[EMAIL PROTECTED]<javascript:;>>
>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I would like to feed a servlet that is serving a long polling request
>> with
>>>> notifications when certain rows in an hbase table change.
>>>>
>>>> It seems that the best mechanism to achieve this would be to add a
>> custom
>>>> RegionObserver. What I'm unsure about though is how to best issue the
>>>> callback. Does hbase provide an RPC mechanism that I could re-use or
>> should
>>>> I consider this a separate problem entirely? Are there alternative ways
>> to
>>>> get notified of row changes that better handle this problem?
>>>>
>>>> Thanks!
>>>> Jan
>>>>
>>
>
>
> --
> Best regards,
>
>   - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

Thanks for the help everyone.
Would you see this as something that could be useful to ship with HBase? If
so, I may well consider what I'm about to write a proof of concept and post
again with my findings / the source.
On 7 February 2013 01:39, Michael Segel <[EMAIL PROTECTED]> wrote:

> I'd say add some sort of durable queue to receive the alert if you don't
> want to miss a message.
>
> Essentially use a region server observer and write the alert to your
> favorite pub/sub solution.
> The key then is starting the pub/sub queue, first so that your region
> server can connect to it.
>
> If you want to get fancy, there's a bit of overkill that you could do...
>  like figure out a way to write the output to HDFS if that made sense.
>
> Just some thoughts before I'm really awake...
>
> -Mike
>
> On Feb 5, 2013, at 10:45 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
>
> > You are better off using your own RPC. Thrift would be a good choice.
> >
> > On Tuesday, February 5, 2013, Jan Althaus wrote:
> >
> >> Sorry for breaking the thread. It seems my subscription wasn't active at
> >> the time the reply was posted.
> >>
> >> The client receiving the notifications would be a separate process,
> >> potentially on a different machine. It could obviously connect to the
> HBase
> >> cluster though.
> >>
> >> I can certainly see how delivering such notifications would be outside
> the
> >> scope of HBase, but I wanted to make sure that I wasn't reinventing the
> >> wheel by using a completely different RPC mechanism between the region
> >> server and the servlet.
> >>
> >>> Please be a little more specific about where the notifications should
> be
> >>> received.
> >>>
> >>> Your custom RegionObserver lives on region server.
> >>>
> >>> I guess you want notifications to be delivered to your client, outside
> >> the
> >>> cluster.
> >>>
> >>> Cheers
> >>>
> >>> On Tue, Feb 5, 2013 at 12:18 PM, Jan Althaus <[EMAIL PROTECTED]
> <javascript:;>>
> >> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I would like to feed a servlet that is serving a long polling request
> >> with
> >>>> notifications when certain rows in an hbase table change.
> >>>>
> >>>> It seems that the best mechanism to achieve this would be to add a
> >> custom
> >>>> RegionObserver. What I'm unsure about though is how to best issue the
> >>>> callback. Does hbase provide an RPC mechanism that I could re-use or
> >> should
> >>>> I consider this a separate problem entirely? Are there alternative
> ways
> >> to
> >>>> get notified of row changes that better handle this problem?
> >>>>
> >>>> Thanks!
> >>>> Jan
> >>>>
> >>
> >
> >
> > --
> > Best regards,
> >
> >   - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
>
>

Yes, I think it could be useful.

For your consideration, it might be interesting to look at HBase
Coprocessor Observer events as instances of Observable<T>:
https://github.com/Netflix/RxJava/blob/master/rxjava-core/src/main/java/rx/Observable.java

On Wed, Feb 6, 2013 at 12:12 PM, Jan Althaus <[EMAIL PROTECTED]> wrote:

> Thanks for the help everyone.
> Would you see this as something that could be useful to ship with HBase? If
> so, I may well consider what I'm about to write a proof of concept and post
> again with my findings / the source.
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Thanks Andrew, I'll look into it.
On 7 February 2013 09:39, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> Yes, I think it could be useful.
>
> For your consideration, it might be interesting to look at HBase
> Coprocessor Observer events as instances of Observable<T>:
>
> https://github.com/Netflix/RxJava/blob/master/rxjava-core/src/main/java/rx/Observable.java
>
>
>
> On Wed, Feb 6, 2013 at 12:12 PM, Jan Althaus <[EMAIL PROTECTED]> wrote:
>
> > Thanks for the help everyone.
> > Would you see this as something that could be useful to ship with HBase?
> If
> > so, I may well consider what I'm about to write a proof of concept and
> post
> > again with my findings / the source.
> >
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>