Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - avoid assigning hbase-access-operatation map/reduce slot to the tasktracker?


Copy link to this message
-
avoid assigning hbase-access-operatation map/reduce slot to the tasktracker?
Jameson Li 2011-03-21, 11:30
Hello,

I have a hadoop cluster that has 10 nodes. I use both of the
hdfs/mapred(0.20.2) and the hbase(0.20.6).
Every node is both of a datanode and a tasktracker.
But I just configured 9 nodes as the region server and the
zookeeper(HBASE_MANAGES_ZK=true).
And then when I run the map/reduce job that access the hbase, it will return
the below error.
Should I avoid assigning hbase-operatation map/reduce slot to the
tasktracker?
Or I have to change all of the nodes as the zookeeper node in all of the
datanode(or tasktracker?)?
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:host.name=aaa*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_20*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.home=....*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.class.path=...*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.library.path=...*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=...*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.name=Linux*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.arch=amd64*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.version=2.6.18-128.el5*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.name=hadoop*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.home=...*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.dir=..*
*2011-03-21 17:17:39,885 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, connectString=localhost:2181 sessionTimeout=60000
watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@6d372656
*
*2011-03-21 17:17:39,901 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:2181*
*......*
*2011-03-21 17:18:44,364 WARN org.apache.zookeeper.ClientCnxn: Session 0x0
for server null, unexpected error, closing socket connection and attempting
reconnect*
*java.net.ConnectException: Connection refused*
* at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)*
* at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)*
* at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)*
*2011-03-21 17:18:44,465 WARN
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to create /hbase
-- check quorum servers, currently=localhost:2181*
*org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase*
* at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)*
* at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)*
* at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:809)*
* at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:837)*
* at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureExists(ZooKeeperWrapper.java:405)
*
* at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureParentExists(ZooKeeperWrapper.java:432)
*
* at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.checkOutOfSafeMode(ZooKeeperWrapper.java:545)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:964)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:625)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:601)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:670)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:630)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:601)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:670)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:634)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:601)
*
* at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:134)*
* at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:112)*
* at com.mediav.hadoop.MVHourlyClickLogImportIntoHBaseMapper.setup(Unknown
Source)*
* at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)*
* at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)*
* at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)*
* at org.apache.hadoop.mapred.Child.main(Child.java:170)*
*2011-03-21 17:18:45,690 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:2181*
*2011-03-21 17:18:45,691 WARN org.apache.zookeeper.ClientCnxn: Session 0x0
for server null, unexpected error, closing socket connection and attempting
reconnect*
*java.net.ConnectException: Connection refused*
* at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)*
* at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)*
* at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)*
*2011-03-21 17:18:46,949 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:2181*
*2011-03-21 17:18:46,950 WARN org.apache.zookeeper.ClientCnxn: Session 0x0
for server null, unexpected error,
+
Harsh J 2011-03-21, 12:08
+
Jameson Li 2011-03-22, 02:45