Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> HBaseStorage looks for local zookeeper instance


Copy link to this message
-
Re: HBaseStorage looks for local zookeeper instance
Hi Hari,

This is a known issue with Pig 0.9.2. Here is how you can fix it:
1) Rebuild your pig-withouthadoop.jar with the patch for PIG-2115.
2) Set "hbase.zookeeper.quorum" to "node1:2181" in
/usr/lib/pig/conf/pig.properties.

Please let me know if you're running CDH4. I can send you
the pig-withouthadoop.jar that includes PIG-2115.

Thanks!
Cheolsoo

On Mon, Aug 6, 2012 at 12:20 AM, Hari Prasanna <[EMAIL PROTECTED]> wrote:

> Hi -
>
> I've been having problems with running Pig(0.9.2) scripts with
> HBase(0.92.1) as source and target.
>
> I'm running a 3 node cluster. Node 1 has the NameNode, JobTracker,
> Zookeeper server and HBase Master
> Node 2 & 3 has the DataNode, TaskTracker and HBase region servers
>
> Pig Script:
>
> A = LOAD 'hbase://tb1' USING
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf1:ssId_20120730
> cf1:arId_20120730','-limit 10') AS (ss_id:int, ar_id:int);
> B = GROUP A BY ss_id;
> C = FOREACH B GENERATE group, COUNT(A) as hits;
> STORE C INTO 'hbase://tbl2' USING
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf1:20120730');
>
> Env variables:
>
> HADOOP_CLASSPATH=.:/usr/lib/hbase/hbase.jar:/etc/hbase/conf
> PIG_CLASSPATH=/usr/lib/hbase/hbase.jar:/usr/lib/hbase
> HBASE_HOME=/usr/lib/hbase
>
>
> The Pig script is able to load the data from HBase(I tried 'dump A'
> which worked fine). However, when trying to store in HBase, it is
> trying to look for a local zookeeper instance at port 2181 and since
> there are no zookeeper instances in the corresponding nodes, the map
> tasks are being killed. I assumed that this is because pig does not
> take the hbase.zookeeper.quorum setting in the hbase config; so I
> tried setting the value in the grunt shell manually (set
> hbase.zookeeper.quorum 'node1'). But, the script still looks for a
> local zookeeper instance.
>
>
> 2012-08-06 01:44:02,367 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server localhost.localdomain/127.0.0.1:2181
> 2012-08-06 01:44:02,367 WARN
> org.apache.zookeeper.client.ZooKeeperSaslClient: SecurityException:
> java.lang.SecurityException: Unable to locate a login configuration
> occurred when trying to find JAAS configuration.
> 2012-08-06 01:44:02,367 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section
> 'Client' could not be found. If you are not using SASL, you may ignore
> this. On the other hand, if you expected SASL to work, please fix your
> JAAS configuration.
> 2012-08-06 01:44:02,368 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x0 for server null, unexpected error, closing socket connection and
> attempting reconnect
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>         at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
>         at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1039)
> 2012-08-06 01:44:02,470 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly
> transient ZooKeeper exception:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2012-08-06 01:44:02,470 ERROR
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper
> exists failed after 3 retries
> 2012-08-06 01:44:02,470 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil:
> hconnection Unable to set watcher on znode /hbase/master
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
>         at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>         at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>         at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
>         at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB