Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Coprocessors and Zookeeper sessions


+
Aaron Tokhy 2012-12-14, 23:54
+
Andrew Purtell 2012-12-15, 18:19
+
Aaron Tokhy 2012-12-17, 16:23
Copy link to this message
-
Re: Coprocessors and Zookeeper sessions
> From the comments it seems like ZooKeeper is now supposed to be
completely hidden from view from the HBase client.

On the client side I think you should just use the ZooKeeper API directly.
You can get the locations of the quorum peers this way:

    Configuration conf = HBaseConfiguration.create();
    String quorum = conf.get(HConstants.ZOOKEEPER_QUORUM);

This should be a comma-separated list of quorum hostnames if the
hbase-site.xml is available and correct.

> This gets me closer to a second option.  I could also store the shared
state in another HBase table (specifically for this task), and use
HTable.incrementColumnValue.  Once the operation is done, I could just
delete the 'session' row from the client side.

My rule of thumb is HBase for data, ZooKeeper for coordination, but this is
only general advice.

Consider how many clients may be coming simultaneously to read that value.
Values from a single location are served by only one RegionServer at a
time, this can become a hot spot if a thundering herd of client requests
arrive. In contrast, you can connect to any ZK quorum peer at random and
optionally stand up ZK servers in observer mode as additional read slaves.
On the other hand, because ZK is a distributed consensus service, in the
worst case any particular client's perspective can lag behind the majority
a bit. (The ZK docs say: "The clients view of the system is guaranteed to
be up-to-date within a certain time bound. (On the order of tens of
seconds.) Either system changes will be seen by a client within this bound,
or the client will detect a service outage.") If you are reading or
updating values out of a HBase table that would not be a consideration.

On Mon, Dec 17, 2012 at 8:23 AM, Aaron Tokhy <[EMAIL PROTECTED]>wrote:

> > You didn't say which HBase version. I'm assuming 0.94.
>
> HBase 0.94.1 and higher.
>
> Looks very simple from the region server side, thanks for the tip.
>
>
>
> As for the client invoking the endpoint based coprocessor, I would like to
> initially set these values so that the region servers would decrement them
> on each finished invocation (until the values are less than zero, which
> signals them to return nothing, and finish).
>
> I could do this through an HConnection instance, but
> HConnection.getZooKeeperWatcher() is deprecated since 0.94.x.  From the
> comments it seems like ZooKeeper is now supposed to be completely hidden
> from view from the HBase client.
>
> This gets me closer to a second option.  I could also store the shared
> state in another HBase table (specifically for this task), and use
> HTable.incrementColumnValue.  Once the operation is done, I could just
> delete the 'session' row from the client side.
>
>
>
> ________________________________________
> From: Andrew Purtell [[EMAIL PROTECTED]]
> Sent: Saturday, December 15, 2012 1:19 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Coprocessors and Zookeeper sessions
>
> You didn't say which HBase version. I'm assuming 0.94.
>
> When registered on a region, your endpoint will get an environment as
> RegionCoprocessorEnvironment. See in
> o.a.h.h.coprocessor.MultiRowMutationEndpoint an example of how an endpoint
> can get the region coprocessor environment, like so:
>
>     RegionCoprocessorEnvironment env = (RegionCoprocessorEnvironment)
> getEnvironment();
>
> You can the get to ZooKeeper via:
>
>    ZooKeeperWatcher watcher = env.getRegionServerServices().getZooKeeper();
>    RecoverableZooKeeper zk = watcher.getRecoverableZooKeeper();
>    ZooKeeper rawZK = zk.getZooKeeper();
>
> Or thereabouts, pardon any typos. The 'zk' object gives you the ZK API with
> HBase's enhancements for retrying idempotent operations. The 'rawZK' object
> is the unadorned ZK ZooKeeper instance.
>
>
> On Fri, Dec 14, 2012 at 3:54 PM, Aaron Tokhy <[EMAIL PROTECTED]
> >wrote:
>
> > Hello HBase users,
> >
> > I would like to store some shared state on every coprocessor invocation
> > (throwing away the values when the coprocessor is finished running).  Is

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB