Hello,

I want to get a hang on implementing a postPut co-processor. A simple
test case is pretty much adding 1 to received cell values and store that
into a second HBase table. I basically have this here:

@Override
public void postPut(
final ObserverContext<RegionCoprocessorEnvironment> c
, final Put put
, final WALEdit edit
, final boolean writeToWAL
) throws IOException {
Map<byte[], List<KeyValue>> familyMap put.getFamilyMap();
RegionCoprocessorEnvironment e = c.getEnvironment();
if
(Arrays.equals(e.getRegion().getTableDesc().getName(),
Bytes.toBytes("t"))) {
List<KeyValue> kvs = familyMap.get("f1");
for (KeyValue kv : kvs) {

}
}
}

Now the question is how to process in the for loop to get the cell
value, increment the value by 1 and store that into a second HBase
table.

In general, I guess instantiating a HBase table should be at a central
point of my region observer class instead of the postPut implementation
etc ... Concrete examples on co-processors are rather rare, the only
useful thing I found were different coprocessor unit test classes, but
they don't go that far.

Thanks!

Thomas

Hi Thomas,

There are some examples in my book, or here https://github.com/larsgeorge/hbase-book/tree/master/ch04/src/main/java/coprocessor. You can use the live cycle methods start() and stop() to create the resources you need. Since the class is instantiated only once this is a common approach to share resources. You need to use the provided CoprocosserEnvironment instance and its getTable() to retrieve the reference.

Lars

On Nov 30, 2011, at 1:44 PM, Steinmaurer Thomas wrote:

> Hello,
>
> I want to get a hang on implementing a postPut co-processor. A simple
> test case is pretty much adding 1 to received cell values and store that
> into a second HBase table. I basically have this here:
>
> @Override
> public void postPut(
> final ObserverContext<RegionCoprocessorEnvironment> c
> , final Put put
> , final WALEdit edit
> , final boolean writeToWAL
> ) throws IOException {
> Map<byte[], List<KeyValue>> familyMap > put.getFamilyMap();
> RegionCoprocessorEnvironment e = c.getEnvironment();
> if
> (Arrays.equals(e.getRegion().getTableDesc().getName(),
> Bytes.toBytes("t"))) {
> List<KeyValue> kvs = familyMap.get("f1");
> for (KeyValue kv : kvs) {
>
> }
> }
> }
>
> Now the question is how to process in the for loop to get the cell
> value, increment the value by 1 and store that into a second HBase
> table.
>
> In general, I guess instantiating a HBase table should be at a central
> point of my region observer class instead of the postPut implementation
> etc ... Concrete examples on co-processors are rather rare, the only
> useful thing I found were different coprocessor unit test classes, but
> they don't go that far.
>
> Thanks!
>
> Thomas
>

Hi Lars,

thanks for jumping in!

Had a look on your example and the RegionObserver examples are all
preGet(). How would you write a prePut coprocessor to read cell values
of the currently processed row, change them (e.g. by adding 1) and let
them pass so that the changed values are written into the original
table.

Thanks,
Thomas

-----Original Message-----
From: Lars George [mailto:[EMAIL PROTECTED]]
Sent: Mittwoch, 30. November 2011 14:36
To: [EMAIL PROTECTED]
Subject: Re: First postPut coprocessor test drive

Hi Thomas,

There are some examples in my book, or here
https://github.com/larsgeorge/hbase-book/tree/master/ch04/src/main/java/
coprocessor. You can use the live cycle methods start() and stop() to
create the resources you need. Since the class is instantiated only once
this is a common approach to share resources. You need to use the
provided CoprocosserEnvironment instance and its getTable() to retrieve
the reference.

Lars

On Nov 30, 2011, at 1:44 PM, Steinmaurer Thomas wrote:

> Hello,
>
> I want to get a hang on implementing a postPut co-processor. A simple
> test case is pretty much adding 1 to received cell values and store
> that into a second HBase table. I basically have this here:
>
> @Override
> public void postPut(
> final ObserverContext<RegionCoprocessorEnvironment> c
> , final Put put
> , final WALEdit edit
> , final boolean writeToWAL
> ) throws IOException {
> Map<byte[], List<KeyValue>> familyMap put.getFamilyMap();
> RegionCoprocessorEnvironment e = c.getEnvironment();
> if
> (Arrays.equals(e.getRegion().getTableDesc().getName(),
> Bytes.toBytes("t"))) {
> List<KeyValue> kvs = familyMap.get("f1");
> for (KeyValue kv : kvs) {
>
> }
> }
> }
>
> Now the question is how to process in the for loop to get the cell
> value, increment the value by 1 and store that into a second HBase
> table.
>
> In general, I guess instantiating a HBase table should be at a central

> point of my region observer class instead of the postPut
> implementation etc ... Concrete examples on co-processors are rather
> rare, the only useful thing I found were different coprocessor unit
> test classes, but they don't go that far.
>
> Thanks!
>
> Thomas
>

Hi Thomas,

You can access and modify the Put instance that is handed into the prePut() callback. If you want to access columns that are *not* in the current Put, then you would need to do a separate get() call to retrieve the values. Adding another column is then accomplished by adding it to the Put instance. Is that what you were asking for?

Lars
On Nov 30, 2011, at 2:46 PM, Steinmaurer Thomas wrote:

> Hi Lars,
>
> thanks for jumping in!
>
> Had a look on your example and the RegionObserver examples are all
> preGet(). How would you write a prePut coprocessor to read cell values
> of the currently processed row, change them (e.g. by adding 1) and let
> them pass so that the changed values are written into the original
> table.
>
> Thanks,
> Thomas
>
> -----Original Message-----
> From: Lars George [mailto:[EMAIL PROTECTED]]
> Sent: Mittwoch, 30. November 2011 14:36
> To: [EMAIL PROTECTED]
> Subject: Re: First postPut coprocessor test drive
>
> Hi Thomas,
>
> There are some examples in my book, or here
> https://github.com/larsgeorge/hbase-book/tree/master/ch04/src/main/java/
> coprocessor. You can use the live cycle methods start() and stop() to
> create the resources you need. Since the class is instantiated only once
> this is a common approach to share resources. You need to use the
> provided CoprocosserEnvironment instance and its getTable() to retrieve
> the reference.
>
> Lars
>
> On Nov 30, 2011, at 1:44 PM, Steinmaurer Thomas wrote:
>
>> Hello,
>>
>> I want to get a hang on implementing a postPut co-processor. A simple
>> test case is pretty much adding 1 to received cell values and store
>> that into a second HBase table. I basically have this here:
>>
>> @Override
>> public void postPut(
>> final ObserverContext<RegionCoprocessorEnvironment> c
>> , final Put put
>> , final WALEdit edit
>> , final boolean writeToWAL
>> ) throws IOException {
>> Map<byte[], List<KeyValue>> familyMap > put.getFamilyMap();
>> RegionCoprocessorEnvironment e = c.getEnvironment();
>> if
>> (Arrays.equals(e.getRegion().getTableDesc().getName(),
>> Bytes.toBytes("t"))) {
>> List<KeyValue> kvs = familyMap.get("f1");
>> for (KeyValue kv : kvs) {
>>
>> }
>> }
>> }
>>
>> Now the question is how to process in the for loop to get the cell
>> value, increment the value by 1 and store that into a second HBase
>> table.
>>
>> In general, I guess instantiating a HBase table should be at a central
>
>> point of my region observer class instead of the postPut
>> implementation etc ... Concrete examples on co-processors are rather
>> rare, the only useful thing I found were different coprocessor unit
>> test classes, but they don't go that far.
>>
>> Thanks!
>>
>> Thomas
>>
>

Thomas.

Just saw your email.

In addition to Lars's response, a couple of things I'd like to point out:

 >> if
 >> (Arrays.equals(e.getRegion().getTableDesc().getName(),
 >> Bytes.toBytes("t"))) {

This piece of code can be eliminated if you only apply the coprocessor
to one table, by setting it at hbase shell:

shell $ alter 't1', METHOD=>'table_attri', 'COPROCESSOR'=>'...'

(i don't have a cluster right now, it may not exactly accurate. )

 >> implementation etc ... Concrete examples on co-processors are rather
 >> rare, the only useful thing I found were different coprocessor unit
 >> test classes, but they don't go that far.

If you check out 0.92 code, ACL of security feature is built ontop of
coprocessor.

Thanks,
Mingjie

On 11/30/2011 05:46 AM, Steinmaurer Thomas wrote:
> Hi Lars,
>
> thanks for jumping in!
>
> Had a look on your example and the RegionObserver examples are all
> preGet(). How would you write a prePut coprocessor to read cell values
> of the currently processed row, change them (e.g. by adding 1) and let
> them pass so that the changed values are written into the original
> table.
>
> Thanks,
> Thomas
>
> -----Original Message-----
> From: Lars George [mailto:[EMAIL PROTECTED]]
> Sent: Mittwoch, 30. November 2011 14:36
> To: [EMAIL PROTECTED]
> Subject: Re: First postPut coprocessor test drive
>
> Hi Thomas,
>
> There are some examples in my book, or here
> https://github.com/larsgeorge/hbase-book/tree/master/ch04/src/main/java/
> coprocessor. You can use the live cycle methods start() and stop() to
> create the resources you need. Since the class is instantiated only once
> this is a common approach to share resources. You need to use the
> provided CoprocosserEnvironment instance and its getTable() to retrieve
> the reference.
>
> Lars
>
> On Nov 30, 2011, at 1:44 PM, Steinmaurer Thomas wrote:
>
>> Hello,
>>
>> I want to get a hang on implementing a postPut co-processor. A simple
>> test case is pretty much adding 1 to received cell values and store
>> that into a second HBase table. I basically have this here:
>>
>> @Override
>> public void postPut(
>> final ObserverContext<RegionCoprocessorEnvironment>  c
>> , final Put put
>> , final WALEdit edit
>> , final boolean writeToWAL
>> ) throws IOException {
>> Map<byte[], List<KeyValue>>  familyMap > put.getFamilyMap();
>> RegionCoprocessorEnvironment e = c.getEnvironment();
>> if
>> (Arrays.equals(e.getRegion().getTableDesc().getName(),
>> Bytes.toBytes("t"))) {
>> List<KeyValue>  kvs = familyMap.get("f1");
>> for (KeyValue kv : kvs) {
>>
>> }
>> }
>> }
>>
>> Now the question is how to process in the for loop to get the cell
>> value, increment the value by 1 and store that into a second HBase
>> table.
>>
>> In general, I guess instantiating a HBase table should be at a central
>
>> point of my region observer class instead of the postPut
>> implementation etc ... Concrete examples on co-processors are rather
>> rare, the only useful thing I found were different coprocessor unit
>> test classes, but they don't go that far.
>>
>> Thanks!
>>
>> Thomas
>>
>
>