Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Problem in using HBase Bulk Load


Copy link to this message
-
Problem in using HBase Bulk Load
Hi All,

I am using cdh3u2(hbase version:0.90.4-cdh3u2).

I am able to perform CRUD operation in Hbase through its shell and On the
web console also it doesn't reports any problem. I also imported data into
HBase with help of Sqoop and it works fine without any problem.

In my cluster i have 3 nodes running tasktracker, datanode,
hbase-regionserver; 1 node running namenode, hbase-master; 1 node running
jobtracker, zookeeper-server.
I have only 1 zookeeper server in my quorum. Following is the configuration
of my zoo.cfg file:
tickTime=2000
dataDir=/var/zookeeper
clientPort=2181
server.0=zookeeper-hostname:2888:3888

The above file is present across the cluster in /etc/zookeeper/ folder.

My hbase-site.xml has the following property for zookeeper:
<property>
  <name>hbase.zookeeper.quorum</name>
  <value>ihub-jobtracker1</value>
</property>
PROBLEM: When i run the bulk load it tries to connect to zookeeper server
at 127.0.0.1:2181. This is incorrect. It should connect to
"zookeeper-hostname:2181". Please provide me pointers in resolving this
problem. Appreciate your help.
Here is the command i run for bulk_loading:
sudo -u hdfs hadoop jar /usr/lib/hbase/hbase-0.90.4-cdh3u2.jar importtsv
-Dimporttsv.columns=HBASE_ROW_KEY,info:APPDATE,info:CANCELREASON,info:CURRENTSTATUS
account_data1 /user/hdfs/account_data1 '-Dimporttsv.separator=,'

Here is the console log after running the above command:
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.3.3-cdh3u2--1, built on 10/14/2011 05:17 GMT
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client environment:host.name
=ihub-namenode1
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_20
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:java.home=/usr/java/jdk1.6.0_20/jre
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=/usr/lib/hadoop-0.20/conf:/usr/java/jdk1.6.0_20/jre//lib/tools.jar:/usr/lib/hadoop-0.20:/usr/lib/hadoop-0.20/hadoop-core-0.20.2-cdh3u2.jar:/usr/lib/hadoop-0.20/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-net-1.4.1.jar:/usr/lib/hadoop-0.20/lib/core-3.1.1.jar:/usr/lib/hadoop-0.20/lib/guava-r06.jar:/usr/lib/hadoop-0.20/lib/hadoop-fairscheduler-0.20.2-cdh3u2.jar:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20/lib/jackson-core-asl-1.5.2.jar:/usr/lib/hadoop-0.20/lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hadoop-0.20/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20/lib/jetty-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jetty-servlet-tester-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jetty-util-6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20/lib/junit-4.5.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20/lib/log4j-1.2.15.jar:/usr/lib/hadoop-0.20/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop-0.20/lib/oro-2.0.8.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-20081211.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop-0.20/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop-0.20/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop-0.20/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20/lib/zookeeper.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1/jsp-api-2.1.jar:/usr/lib/hadoop/lib:/usr/lib/hbase/lib:/usr/lib/sqoop/lib
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=/usr/java/jdk1.6.0_20/jre/lib/amd64/server:/usr/java/jdk1.6.0_20/jre/lib/amd64:/usr/java/jdk1.6.0_20/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:os.version=2.6.32-71.el6.x86_64
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client environment:user.name
=hdfs
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:user.home=/usr/lib/hadoop-0.20
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/etc/hbase/conf
12/03/02 09:40:27 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
12/03/02 09:40:27 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost/127.0.0.1:2181
12/03/02 09:40:27 WARN zookeeper.ClientCnxn: Session 0x0 for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
12/03/02 09:40:28 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost/127.0.0.1:2181
12/03/02 09:40:28 WARN zookeeper.ClientCnxn: Session 0x0 for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB