Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> issure about HA in hadoop name node


Copy link to this message
-
issure about HA in hadoop name node
hi ,all:
i deploy HA on hadoop name node ,and test it ,autoFailover is ok ,but when
i add HA to hbase ,it cause two name node both in standby mode,and i try to
failover manually ,but failed ,why? (BTW: i use CDH4.3,and hbase is 0.94.6)

# sudo -u hdfs hdfs haadmin -DfSHAadmin -failover nn2 nn1
Operation failed: Call From node1/192.168.142.129 to node2:8019 failed on
socket timeout exception: java.net.SocketTimeoutException: 10000 millis
timeout while waiting for channel to be ready for read. ch :
java.nio.channels.SocketChannel[connected
local=/192.168.142.129:54094remote=node2/
192.168.142.130:8019]; For more details see:
http://wiki.apache.org/hadoop/SocketTimeout
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:782)
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:748)
        at org.apache.hadoop.ipc.Client.call(Client.java:1229)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
        at com.sun.proxy.$Proxy9.cedeActive(Unknown Source)
        at
org.apache.hadoop.ha.protocolPB.ZKFCProtocolClientSideTranslatorPB.cedeActive(ZKFCProtocolClientSideTranslatorPB.java:64)
        at
org.apache.hadoop.ha.ZKFailoverController.doGracefulFailover(ZKFailoverController.java:642)
        at
org.apache.hadoop.ha.ZKFailoverController.access$400(ZKFailoverController.java:58)
        at
org.apache.hadoop.ha.ZKFailoverController$3.run(ZKFailoverController.java:591)
        at
org.apache.hadoop.ha.ZKFailoverController$3.run(ZKFailoverController.java:588)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at
org.apache.hadoop.ha.ZKFailoverController.gracefulFailoverToYou(ZKFailoverController.java:588)
        at
org.apache.hadoop.ha.ZKFCRpcServer.gracefulFailover(ZKFCRpcServer.java:94)
        at
org.apache.hadoop.ha.protocolPB.ZKFCProtocolServerSideTranslatorPB.gracefulFailover(ZKFCProtocolServerSideTranslatorPB.java:61)
        at
org.apache.hadoop.ha.proto.ZKFCProtocolProtos$ZKFCProtocolService$2.callBlockingMethod(ZKFCProtocolProtos.java:1351)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)
Caused by: java.net.SocketTimeoutException: 10000 millis timeout while
waiting for channel to be ready for read. ch :
java.nio.channels.SocketChannel[connected
local=/192.168.142.129:54094remote=node2/
192.168.142.130:8019]
        at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:165)
        at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:156)
        at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:129)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at
org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:409)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
        at java.io.FilterInputStream.read(FilterInputStream.java:83)
        at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:276)
        at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:760)
        at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:288)
        at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:752)
        at
org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:985)
        at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:938)
        at org.apache.hadoop.ipc.Client$Connection.run(Client.java:836)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB