Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Zookeeper exception while trying to access HBase table using pig.


Copy link to this message
-
Zookeeper exception while trying to access HBase table using pig.
Hi,
I have a latest pig build from trunk. I have configured it to run on 12-node
hadoop cluster.
I am trying to access a hbase table, the map job is running fine for
sometime...but after some time.. I am getting zookeeper exception

hadoop@ub13:/usr/local/hadoop/hbase/conf$ pig
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.conf.Configuration).
log4j:WARN Please initialize the log4j system properly.
2011-07-05 12:40:23,737 [main] INFO  org.apache.pig.Main - Logging error
messages to: /usr/local/hadoop/hbase/conf/pig_1309849823721.log
2011-07-05 12:40:24,047 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: hdfs://ub13:54310
2011-07-05 12:40:24,241 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to map-reduce job tracker at: ub13:54311

grunt> ht = load 'hbase://tweetinfo' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('prim_tweetinfo:name',
'-loadKey') as (id,name);
grunt> dump ht | HEAD 100;

2011-07-05 12:40:38,234 [main] INFO
org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script:
UNKNOWN
2011-07-05 12:40:38,481 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
File concatenation threshold: 100 optimistic? false
2011-07-05 12:40:38,513 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 1
2011-07-05 12:40:38,514 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 1
2011-07-05 12:40:38,559 [main] INFO
org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to
the job
2011-07-05 12:40:38,572 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:host.name=ub13
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.version=1.6.0_20
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.vendor=Sun Microsystems Inc.
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.home=/usr/lib/jvm/java-6-openjdk/jre
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.class.path=/usr/local/hadoop/pig-0.10/bin/../conf:/usr/lib/jvm/java-6-openjdk/lib/tools.jar:/usr/local/hadoop/pig-0.10/bin/../build/classes:/usr/local/hadoop/pig-0.10/bin/../build/test/classes:/usr/local/hadoop/pig-0.10/bin/../pig-*-core.jar:/usr/local/hadoop/pig-0.10/bin/../build/pig-0.10.0-SNAPSHOT.jar:/usr/local/hadoop/pig-0.10/bin/../lib/automaton.jar:/usr/local/hadoop/hadoop/hadoop-0.20-append-for-hbase-core.jar:/usr/local/hadoop/hbase/hbase-0.90.3.jar:/usr/local/hadoop/hbase/hbase-0.90.3-tests.jar:/usr/local/hadoop/hbase/lib/zookeeper-3.3.2.jar:/usr/local/hadoop/hadoop:/usr/local/hadoop/hadoop/conf
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.library.path=/usr/lib/jvm/java-6-openjdk/jre/lib/i386/server:/usr/lib/jvm/java-6-openjdk/jre/lib/i386:/usr/lib/jvm/java-6-openjdk/jre/../lib/i386:/usr/java/packages/lib/i386:/usr/lib/jni:/lib:/usr/lib
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.io.tmpdir=/tmp
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:java.compiler=<NA>
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:os.name=Linux
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:os.arch=i386
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:os.version=2.6.35-22-generic-pae
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:user.name=hadoop
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:user.home=/home/hadoop
2011-07-05 12:40:38,737 [main] INFO  org.apache.zookeeper.ZooKeeper - Client
environment:user.dir=/usr/local/hadoop/hbase/conf
2011-07-05 12:40:38,739 [main] INFO  org.apache.zookeeper.ZooKeeper -
Initiating client connection, connectString=localhost:2181
sessionTimeout=180000 watcher=hconnection
2011-07-05 12:40:38,791 [main-SendThread()] INFO
org.apache.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181
2011-07-05 12:40:38,814 [main-SendThread(localhost:2181)] INFO
org.apache.zookeeper.ClientCnxn - Socket connection established to
localhost/127.0.0.1:2181, initiating session
2011-07-05 12:40:38,833 [main-SendThread(localhost:2181)] INFO
org.apache.zookeeper.ClientCnxn - Session establishment complete on server
localhost/127.0.0.1:2181, sessionid = 0x30f91a0a2c0009, negotiated timeout 180000
2011-07-05 12:40:43,484 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2011-07-05 12:40:43,529 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
2011-07-05 12:40:44,030 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2011-07-05 12:40:44,499 [Thread-13] INFO  org.apache.zookeeper.ZooKeeper -
Initiating client connection, connectString=localhost:2181
sessionTimeout=180000 watcher=hconnection
2011-07-05 12:40:44,500 [Thread-13-SendThread()] INFO
org.apache.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:2181
2011-07-05 12:40:44,501 [Thread-13-SendThread(localhost:2181)] INFO
org.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB