Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Zookeeper exception while trying to access HBase table using pig.


Copy link to this message
-
Zookeeper exception while trying to access HBase table using pig.
Hi,
I have a latest pig build from trunk. I have configured it to run on 12-node
hadoop cluster.
I am trying to access a hbase table, the map job is running fine for
sometime...but after some time.. I am getting zookeeper exception

hadoop@ub13:/usr/local/hadoop/hbase/conf$ pig
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.conf.Configuration).
log4j:WARN Please initialize the log4j system properly.
2011-07-05 12:40:23,737 [main] INFO  org.apache.pig.Main - Logging error
messages to: /usr/local/hadoop/hbase/conf/pig_1309849823721.log
2011-07-05 12:40:24,047 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: hdfs://ub13:54310
2011-07-05 12:40:24,241 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to map-reduce job tracker at: ub13:54311

grunt> ht = load 'hbase://tweetinfo' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('prim_tweetinfo:name',
'-loadKey') as (id,name);
grunt> dump ht | HEAD 100;

2011-07-05 12:40:38,234 [main] INFO
org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script:
UNKNOWN
2011-07-05 12:40:38,481 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
File concatenation threshold: 100 optimistic? false
2011-07-05 12:40:38,513 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 1
2011-07-05 12:40:38,514 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 1
2011-07-05 12:40:38,559 [main] INFO
org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to
the job
2011-07-05 12:40:38,572 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:host.name=ub13
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.version=1.6.0_20
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.vendor=Sun Microsystems Inc.
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.home=/usr/lib/jvm/java-6-openjdk/jre
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.class.path=/usr/local/hadoop/pig-0.10/bin/../conf:/usr/lib/jvm/java-6-openjdk/lib/tools.jar:/usr/local/hadoop/pig-0.10/bin/../build/classes:/usr/local/hadoop/pig-0.10/bin/../build/test/classes:/usr/local/hadoop/pig-0.10/bin/../pig-*-core.jar:/usr/local/hadoop/pig-0.10/bin/../build/pig-0.10.0-SNAPSHOT.jar:/usr/local/hadoop/pig-0.10/bin/../lib/automaton.jar:/usr/local/hadoop/hadoop/hadoop-0.20-append-for-hbase-core.jar:/usr/local/hadoop/hbase/hbase-0.90.3.jar:/usr/local/hadoop/hbase/hbase-0.90.3-tests.jar:/usr/local/hadoop/hbase/lib/zookeeper-3.3.2.jar:/usr/local/hadoop/hadoop:/usr/local/hadoop/hadoop/conf
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.library.path=/usr/lib/jvm/java-6-openjdk/jre/lib/i386/server:/usr/lib/jvm/java-6-openjdk/jre/lib/i386:/usr/lib/jvm/java-6-openjdk/jre/../lib/i386:/usr/java/packages/lib/i386:/usr/lib/jni:/lib:/usr/lib
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.io.tmpdir=/tmp
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.compiler=<NA>
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:os.name=Linux
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:os.arch=i386
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:os.version=2.6.35-22-generic-pae
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:user.name=hadoop
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:user.home=/home/hadoop
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:user.dir=/usr/local/hadoop/hbase/conf
2011-07-05 12:40:38,739 [main] INFO  org.apache.zookeeper.ZooKeeper -
Initiating client connection, connectString=localhost:2181
sessionTimeout=180000 watcher=hconnection
2011-07-05 12:40:38,791 [main-SendThread()] INFO
org.apache.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181
2011-07-05 12:40:38,814 [main-SendThread(localhost:2181)] INFO
org.apache.zookeeper.ClientCnxn - Socket connection established to
localhost/127.0.0.1:2181, initiating session
2011-07-05 12:40:38,833 [main-SendThread(localhost:2181)] INFO
org.apache.zookeeper.ClientCnxn - Session establishment complete on server
localhost/127.0.0.1:2181, sessionid = 0x30f91a0a2c0009, negotiated timeout 180000
2011-07-05 12:40:43,484 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2011-07-05 12:40:43,529 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
2011-07-05 12:40:44,030 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2011-07-05 12:40:44,499 [Thread-13] INFO  org.apache.zookeeper.ZooKeeper -
Initiating client connection, connectString=localhost:2181
sessionTimeout=180000 watcher=hconnection
2011-07-05 12:40:44,500 [Thread-13-SendThread()] INFO
org.apache.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181
2011-07-05 12:40:44,501 [Thread-13-SendThread(localhost:2181)] INFO
org.