Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - issure about HA in hadoop name node


Copy link to this message
-
issure about HA in hadoop name node
ch huang 2013-08-05, 04:56
hi ,all:
i deploy HA on hadoop name node ,and test it ,autoFailover is ok ,but when
i add HA to hbase ,it cause two name node both in standby mode,and i try to
failover manually ,but failed ,why? (BTW: i use CDH4.3,and hbase is 0.94.6)

# sudo -u hdfs hdfs haadmin -DfSHAadmin -failover nn2 nn1
Operation failed: Call From node1/192.168.142.129 to node2:8019 failed on
socket timeout exception: java.net.SocketTimeoutException: 10000 millis
timeout while waiting for channel to be ready for read. ch :
java.nio.channels.SocketChannel[connected
local=/192.168.142.129:54094remote=node2/
192.168.142.130:8019]; For more details see:
http://wiki.apache.org/hadoop/SocketTimeout
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:782)
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:748)
        at org.apache.hadoop.ipc.Client.call(Client.java:1229)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
        at com.sun.proxy.$Proxy9.cedeActive(Unknown Source)
        at
org.apache.hadoop.ha.protocolPB.ZKFCProtocolClientSideTranslatorPB.cedeActive(ZKFCProtocolClientSideTranslatorPB.java:64)
        at
org.apache.hadoop.ha.ZKFailoverController.doGracefulFailover(ZKFailoverController.java:642)
        at
org.apache.hadoop.ha.ZKFailoverController.access$400(ZKFailoverController.java:58)
        at
org.apache.hadoop.ha.ZKFailoverController$3.run(ZKFailoverController.java:591)
        at
org.apache.hadoop.ha.ZKFailoverController$3.run(ZKFailoverController.java:588)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at
org.apache.hadoop.ha.ZKFailoverController.gracefulFailoverToYou(ZKFailoverController.java:588)
        at
org.apache.hadoop.ha.ZKFCRpcServer.gracefulFailover(ZKFCRpcServer.java:94)
        at
org.apache.hadoop.ha.protocolPB.ZKFCProtocolServerSideTranslatorPB.gracefulFailover(ZKFCProtocolServerSideTranslatorPB.java:61)
        at
org.apache.hadoop.ha.proto.ZKFCProtocolProtos$ZKFCProtocolService$2.callBlockingMethod(ZKFCProtocolProtos.java:1351)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)
Caused by: java.net.SocketTimeoutException: 10000 millis timeout while
waiting for channel to be ready for read. ch :
java.nio.channels.SocketChannel[connected
local=/192.168.142.129:54094remote=node2/
192.168.142.130:8019]
        at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:165)
        at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:156)
        at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:129)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at
org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:409)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
        at java.io.FilterInputStream.read(FilterInputStream.java:83)
        at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:276)
        at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:760)
        at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:288)
        at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:752)
        at
org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:985)
        at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:938)
        at org.apache.hadoop.ipc.Client$Connection.run(Client.java:836)