Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - DFSClient not able to write


Copy link to this message
-
DFSClient not able to write
Subroto 2012-06-19, 08:29
Hi,

When DFS cluster;
The datanodes keep on dumping the logs:
2012-06-19 04:05:04,842 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding block pool BP-1233167859-10.10.35.8-1340093005960
2012-06-19 04:05:04,861 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-1233167859-10.10.35.8-1340093005960 (storage id DS-1255092928-10.244.15.192-50010-1340093104449) service to ec2-107-22-39-220.compute-1.amazonaws.com/10.10.35.8:9000 beginning handshake with NN
2012-06-19 04:05:04,979 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool Block pool BP-1233167859-10.10.35.8-1340093005960 (storage id DS-1255092928-10.244.15.192-50010-1340093104449) service to ec2-107-22-39-220.compute-1.amazonaws.com/10.10.35.8:9000 successfully registered with NN
2012-06-19 04:05:04,980 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: For namenode ec2-107-22-39-220.compute-1.amazonaws.com/10.10.35.8:9000 using DELETEREPORT_INTERVAL of 300000 msec  BLOCKREPORT_INTERVAL of 21600000msec Initial delay: 0msec; heartBeatInterval=3000
2012-06-19 04:05:05,258 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Namenode Block pool BP-1233167859-10.10.35.8-1340093005960 (storage id DS-1255092928-10.244.15.192-50010-1340093104449) service to ec2-107-22-39-220.compute-1.amazonaws.com/10.10.35.8:9000 trying to claim ACTIVE state with txid=24
2012-06-19 04:05:05,258 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Acknowledging ACTIVE Namenode Block pool BP-1233167859-10.10.35.8-1340093005960 (storage id DS-1255092928-10.244.15.192-50010-1340093104449) service to ec2-107-22-39-220.compute-1.amazonaws.com/10.10.35.8:9000
2012-06-19 04:05:05,394 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 0 blocks took 1 msec to generate and 135 msecs for RPC and NN processing
2012-06-19 04:05:05,394 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: sent block report, processed command:org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@49d31859
2012-06-19 04:05:05,396 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Periodic Block Verification Scanner initialized with interval 504 hours for block pool BP-1233167859-10.10.35.8-1340093005960.
2012-06-19 04:05:05,400 INFO org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Added bpid=BP-1233167859-10.10.35.8-1340093005960 to blockPoolScannerMap, new size=1
2012-06-19 04:05:09,845 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%
2012-06-19 04:05:14,848 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%
2012-06-19 04:05:19,849 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%
2012-06-19 04:05:24,850 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%
2012-06-19 04:05:29,851 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%
2012-06-19 04:05:34,852 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%
2012-06-19 04:05:39,853 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%
2012-06-19 04:05:44,854 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%
2012-06-19 04:05:49,855 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Starting a new period : work left in prev period : 0.00%

The DFSClient operations are also not taking place properly:
2012-06-19 04:04:56,653 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception
java.io.IOException: File /hbase/hbase.version could only be replicated to 0 nodes instead of minReplication (=1).  There are 1 datanode(s) running and no node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1256)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1977)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:470)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:292)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:42602)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)

Please let me know if there is any configuration which needs to be set…..
The hadoop version being used is: 2.0.0

Cheers,
Subroto Sanyal