Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> avoid assigning hbase-access-operatation map/reduce slot to the tasktracker?


Copy link to this message
-
avoid assigning hbase-access-operatation map/reduce slot to the tasktracker?
Hello,

I have a hadoop cluster that has 10 nodes. I use both of the
hdfs/mapred(0.20.2) and the hbase(0.20.6).
Every node is both of a datanode and a tasktracker.
But I just configured 9 nodes as the region server and the
zookeeper(HBASE_MANAGES_ZK=true).
And then when I run the map/reduce job that access the hbase, it will return
the below error.
Should I avoid assigning hbase-operatation map/reduce slot to the
tasktracker?
Or I have to change all of the nodes as the zookeeper node in all of the
datanode(or tasktracker?)?
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:host.name=aaa*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_20*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.home=....*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.class.path=...*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.library.path=...*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=...*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.name=Linux*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.arch=amd64*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.version=2.6.18-128.el5*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.name=hadoop*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.home=...*
*2011-03-21 17:17:39,883 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.dir=..*
*2011-03-21 17:17:39,885 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, connectString=localhost:2181 sessionTimeout=60000
watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@6d372656
*
*2011-03-21 17:17:39,901 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:2181*
*......*
*2011-03-21 17:18:44,364 WARN org.apache.zookeeper.ClientCnxn: Session 0x0
for server null, unexpected error, closing socket connection and attempting
reconnect*
*java.net.ConnectException: Connection refused*
* at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)*
* at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)*
* at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)*
*2011-03-21 17:18:44,465 WARN
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to create /hbase
-- check quorum servers, currently=localhost:2181*
*org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase*
* at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)*
* at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)*
* at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:809)*
* at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:837)*
* at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureExists(ZooKeeperWrapper.java:405)
*
* at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureParentExists(ZooKeeperWrapper.java:432)
*
* at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.checkOutOfSafeMode(ZooKeeperWrapper.java:545)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:964)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:625)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:601)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:670)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:630)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:601)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:670)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:634)
*
* at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:601)
*
* at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:134)*
* at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:112)*
* at com.mediav.hadoop.MVHourlyClickLogImportIntoHBaseMapper.setup(Unknown
Source)*
* at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)*
* at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)*
* at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)*
* at org.apache.hadoop.mapred.Child.main(Child.java:170)*
*2011-03-21 17:18:45,690 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:2181*
*2011-03-21 17:18:45,691 WARN org.apache.zookeeper.ClientCnxn: Session 0x0
for server null, unexpected error, closing socket connection and attempting
reconnect*
*java.net.ConnectException: Connection refused*
* at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)*
* at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)*
* at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)*
*2011-03-21 17:18:46,949 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:2181*
*2011-03-21 17:18:46,950 WARN org.apache.zookeeper.ClientCnxn: Session 0x0
for server null, unexpected error,
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB