Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Can a RegionObserver coprocessor increment counter of a row key that may not belong to the region ?


Copy link to this message
-
Re: Can a RegionObserver coprocessor increment counter of a row key that may not belong to the region ?
Toby Lazar 2012-12-06, 16:25
Does this exclude the use of coprocessors to update secondary indexes
(since they will likely be in a different table/region)?  Is there a
recommended approach to real-time secondary indexes that does not use
coprocessors?

Thanks,

Toby

On 12/6/12, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> If you as a coprocessor are not writing back to your own region, you are
> probably doing it wrong.
>
> On Thursday, December 6, 2012, Amit Sela wrote:
>
>> If I'm aggregating over the same Increment Object that triggered the
>> Observer (the same key, same CFs but aggregating into different
>> qualifiers), do I go outside the Region ?
>> I know that for the cases I mentioned earlier I do but in the case I
>> describe here don't I stay in the region ?
>>
>>
>> On Thu, Dec 6, 2012 at 9:40 AM, Andrew Purtell
>> <[EMAIL PROTECTED]<javascript:;>>
>> wrote:
>>
>> > Summing/aggregating with a Coprocessor seems like a fine use, but it
>> > should act upon only the data within the region to which it is
>> > associated.
>> >
>> > However if it seems your Coprocessor must unavoidably update or access
>> > data outside of its region, then either your schema design should
>> > change (such that all data the coprocessor will operate on is
>> > contained within the keyspace of the region); or, you should exclude
>> > all possible client side options. For example maybe Datacube will work
>> > for you here? See http://github.com/urbanairship/datacube
>> >
>> > On 12/6/12, Amit Sela <[EMAIL PROTECTED] <javascript:;>> wrote:
>> > > I asked because I thought exactly what you all said but wanted to be
>> > sure.
>> > > My case is incrementing counters per URL where the families are time
>> > > periods (hourly, daily...) and the qualifiers are
>> > > impressions_criteria1,impressions_criteria2... I am going to use
>> > > coprocessors in order to sum all impressions counters (all criteria
>> > > segments) and then increment another counter (general count) that
>> belongs
>> > > to the same URL.
>> > > Is that a good practice for Coprocessors (Observer) ?
>> > >
>> > > Thanks.
>> > >
>> > >
>> > > On Wed, Dec 5, 2012 at 8:56 PM, Asaf Mesika
>> > > <[EMAIL PROTECTED]<javascript:;>
>> >
>> > wrote:
>> > >
>> > >> Why not simply send Increment objects from the client side to HBase,
>> to
>> > >> the URL row key and to the domain row key?
>> > >>
>> > >> On 5 בדצמ 2012, at 14:35, Amit Sela
>> > >> <[EMAIL PROTECTED]<javascript:;>>
>> wrote:
>> > >>
>> > >> > Hi all,
>> > >> >
>> > >> > I have to count the occurrence of URLs in our system using the URL
>> as
>> > >> > row
>> > >> > key and using Increment.
>> > >> > I also want to count domains, so I was thinking of using
>> > RegionObserver
>> > >> > coprocessor on postIncrement() and Increment using the domain as
>> key.
>> > >> >
>> > >> > Will that work ? Is there a problem if the url key starts with
>> 'www.'
>> > >> > and
>> > >> > the domain starts with 'aaaa' ?
>> > >> >
>> > >> > Thanks,
>> > >> >
>> > >> > Amit.
>> > >>
>> > >>
>> > >
>> >
>>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

--
Sent from my mobile device