Hello HBase users,

I would like to store some shared state on every coprocessor invocation (throwing away the values when the coprocessor is finished running).  Is there a way to grab a handle of the regionserver's zookeeper session through a endpoint based coprocessor to rely on the said state?  I just want to store 2 atomic long values used by the coprocessors to tell them to finish running.

-Thanks
Aaron Tokhy

You didn't say which HBase version. I'm assuming 0.94.

When registered on a region, your endpoint will get an environment as
RegionCoprocessorEnvironment. See in
o.a.h.h.coprocessor.MultiRowMutationEndpoint an example of how an endpoint
can get the region coprocessor environment, like so:

    RegionCoprocessorEnvironment env = (RegionCoprocessorEnvironment)
getEnvironment();

You can the get to ZooKeeper via:

   ZooKeeperWatcher watcher = env.getRegionServerServices().getZooKeeper();
   RecoverableZooKeeper zk = watcher.getRecoverableZooKeeper();
   ZooKeeper rawZK = zk.getZooKeeper();

Or thereabouts, pardon any typos. The 'zk' object gives you the ZK API with
HBase's enhancements for retrying idempotent operations. The 'rawZK' object
is the unadorned ZK ZooKeeper instance.
On Fri, Dec 14, 2012 at 3:54 PM, Aaron Tokhy <[EMAIL PROTECTED]>wrote:

> Hello HBase users,
>
> I would like to store some shared state on every coprocessor invocation
> (throwing away the values when the coprocessor is finished running).  Is
> there a way to grab a handle of the regionserver's zookeeper session
> through a endpoint based coprocessor to rely on the said state?  I just
> want to store 2 atomic long values used by the coprocessors to tell them to
> finish running.
>
> -Thanks
> Aaron Tokhy
--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

> You didn't say which HBase version. I'm assuming 0.94.

HBase 0.94.1 and higher.

Looks very simple from the region server side, thanks for the tip.

As for the client invoking the endpoint based coprocessor, I would like to initially set these values so that the region servers would decrement them on each finished invocation (until the values are less than zero, which signals them to return nothing, and finish).

I could do this through an HConnection instance, but HConnection.getZooKeeperWatcher() is deprecated since 0.94.x.  From the comments it seems like ZooKeeper is now supposed to be completely hidden from view from the HBase client.

This gets me closer to a second option.  I could also store the shared state in another HBase table (specifically for this task), and use HTable.incrementColumnValue.  Once the operation is done, I could just delete the 'session' row from the client side.

________________________________________
From: Andrew Purtell [[EMAIL PROTECTED]]
Sent: Saturday, December 15, 2012 1:19 PM
To: [EMAIL PROTECTED]
Subject: Re: Coprocessors and Zookeeper sessions

You didn't say which HBase version. I'm assuming 0.94.

When registered on a region, your endpoint will get an environment as
RegionCoprocessorEnvironment. See in
o.a.h.h.coprocessor.MultiRowMutationEndpoint an example of how an endpoint
can get the region coprocessor environment, like so:

    RegionCoprocessorEnvironment env = (RegionCoprocessorEnvironment)
getEnvironment();

You can the get to ZooKeeper via:

   ZooKeeperWatcher watcher = env.getRegionServerServices().getZooKeeper();
   RecoverableZooKeeper zk = watcher.getRecoverableZooKeeper();
   ZooKeeper rawZK = zk.getZooKeeper();

Or thereabouts, pardon any typos. The 'zk' object gives you the ZK API with
HBase's enhancements for retrying idempotent operations. The 'rawZK' object
is the unadorned ZK ZooKeeper instance.
On Fri, Dec 14, 2012 at 3:54 PM, Aaron Tokhy <[EMAIL PROTECTED]>wrote:

> Hello HBase users,
>
> I would like to store some shared state on every coprocessor invocation
> (throwing away the values when the coprocessor is finished running).  Is
> there a way to grab a handle of the regionserver's zookeeper session
> through a endpoint based coprocessor to rely on the said state?  I just
> want to store 2 atomic long values used by the coprocessors to tell them to
> finish running.
>
> -Thanks
> Aaron Tokhy
--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

> From the comments it seems like ZooKeeper is now supposed to be
completely hidden from view from the HBase client.

On the client side I think you should just use the ZooKeeper API directly.
You can get the locations of the quorum peers this way:

    Configuration conf = HBaseConfiguration.create();
    String quorum = conf.get(HConstants.ZOOKEEPER_QUORUM);

This should be a comma-separated list of quorum hostnames if the
hbase-site.xml is available and correct.

> This gets me closer to a second option.  I could also store the shared
state in another HBase table (specifically for this task), and use
HTable.incrementColumnValue.  Once the operation is done, I could just
delete the 'session' row from the client side.

My rule of thumb is HBase for data, ZooKeeper for coordination, but this is
only general advice.

Consider how many clients may be coming simultaneously to read that value.
Values from a single location are served by only one RegionServer at a
time, this can become a hot spot if a thundering herd of client requests
arrive. In contrast, you can connect to any ZK quorum peer at random and
optionally stand up ZK servers in observer mode as additional read slaves.
On the other hand, because ZK is a distributed consensus service, in the
worst case any particular client's perspective can lag behind the majority
a bit. (The ZK docs say: "The clients view of the system is guaranteed to
be up-to-date within a certain time bound. (On the order of tens of
seconds.) Either system changes will be seen by a client within this bound,
or the client will detect a service outage.") If you are reading or
updating values out of a HBase table that would not be a consideration.

On Mon, Dec 17, 2012 at 8:23 AM, Aaron Tokhy <[EMAIL PROTECTED]>wrote:

> > You didn't say which HBase version. I'm assuming 0.94.
>
> HBase 0.94.1 and higher.
>
> Looks very simple from the region server side, thanks for the tip.
>
>
>
> As for the client invoking the endpoint based coprocessor, I would like to
> initially set these values so that the region servers would decrement them
> on each finished invocation (until the values are less than zero, which
> signals them to return nothing, and finish).
>
> I could do this through an HConnection instance, but
> HConnection.getZooKeeperWatcher() is deprecated since 0.94.x.  From the
> comments it seems like ZooKeeper is now supposed to be completely hidden
> from view from the HBase client.
>
> This gets me closer to a second option.  I could also store the shared
> state in another HBase table (specifically for this task), and use
> HTable.incrementColumnValue.  Once the operation is done, I could just
> delete the 'session' row from the client side.
>
>
>
> ________________________________________
> From: Andrew Purtell [[EMAIL PROTECTED]]
> Sent: Saturday, December 15, 2012 1:19 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Coprocessors and Zookeeper sessions
>
> You didn't say which HBase version. I'm assuming 0.94.
>
> When registered on a region, your endpoint will get an environment as
> RegionCoprocessorEnvironment. See in
> o.a.h.h.coprocessor.MultiRowMutationEndpoint an example of how an endpoint
> can get the region coprocessor environment, like so:
>
>     RegionCoprocessorEnvironment env = (RegionCoprocessorEnvironment)
> getEnvironment();
>
> You can the get to ZooKeeper via:
>
>    ZooKeeperWatcher watcher = env.getRegionServerServices().getZooKeeper();
>    RecoverableZooKeeper zk = watcher.getRecoverableZooKeeper();
>    ZooKeeper rawZK = zk.getZooKeeper();
>
> Or thereabouts, pardon any typos. The 'zk' object gives you the ZK API with
> HBase's enhancements for retrying idempotent operations. The 'rawZK' object
> is the unadorned ZK ZooKeeper instance.
>
>
> On Fri, Dec 14, 2012 at 3:54 PM, Aaron Tokhy <[EMAIL PROTECTED]
> >wrote:
>
> > Hello HBase users,
> >
> > I would like to store some shared state on every coprocessor invocation
> > (throwing away the values when the coprocessor is finished running).  Is

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)