Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> problem about starting datanode


+
ch huang 2013-07-26, 05:31
Copy link to this message
-
Re: problem about starting datanode
You have one datanode data volume (dfs.datanode.data.dir) configured,
but you've specified tolerated failures as 3. The state of 3 > 1 is
invalid and hence the error. You cannot enable disk failure toleration
of a DN with just one volume, so remove the toleration config and your
problem will be resolved.

On Fri, Jul 26, 2013 at 11:01 AM, ch huang <[EMAIL PROTECTED]> wrote:
> i config name node HA,but when i start data node ,i found the error info in
> log
> and here is my hdfs-site.xml file
>
> <configuration>
> <!--
>   <property>
>      <name>dfs.name.dir</name>
>      <value>/var/lib/hadoop-hdfs/cache/hdfs/dfs/name</value>
>   </property>
> -->
> <property>
>  <name>dfs.permissions.superusergroup</name>
>  <value>hadoop</value>
> </property>
> <!--
> <property>
>         <name>dfs.namenode.name.dir</name>
>         <value>/data/hadoopnamespace</value>
> </property>
> -->
> <property>
>         <name>dfs.datanode.data.dir</name>
>         <value>/data/hadoopdataspace</value>
> </property>
> <property>
>         <name>dfs.datanode.failed.volumes.tolerated</name>
>         <value>3</value>
> </property>
> <!-- HA -->
> <property>
>         <name>dfs.nameservices</name>
>         <value>mycluster</value>
> </property>
> <property>
>         <name>dfs.ha.namenodes.mycluster</name>
>         <value>nn1,nn2</value>
> </property>
> <property>
>         <name>dfs.namenode.rpc-address.mycluster.nn1</name>
>         <value>node1:8020</value>
> </property>
> <property>
>         <name>dfs.namenode.rpc-address.mycluster.nn2</name>
>         <value>node2:8020</value>
> </property>
> <property>
>         <name>dfs.namenode.http-address.mycluster.nn1</name>
>         <value>node1:50070</value>
> </property>
> <property>
>         <name>dfs.namenode.http-address.mycluster.nn2</name>
>         <value>node2:50070</value>
> </property>
> <property>
>         <name>dfs.namenode.shared.edits.dir</name>
>         <value>qjournal://node1:8485;node2:8485;node3:8485/mycluster</value>
> </property>
> <property>
>         <name>dfs.journalnode.edits.dir</name>
>         <value>/data/1/dfs/jn</value>
> </property>
> <property>
>         <name>dfs.client.failover.proxy.provider.mycluster</name>
>
> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
> </property>
> <!-- HA end -->
> <!-- ssh fence method -->
> <property>
>         <name>dfs.ha.fencing.methods</name>
>         <value>sshfence</value>
> </property>
> <property>
>         <name>dfs.ha.fencing.ssh.private-key-files</name>
>         <value>/home/nodefence/.ssh/id_rsa</value>
> </property>
> <property>
>   <name>dfs.ha.fencing.ssh.connect-timeout</name>
>   <value>30000</value>
>   <description>
>     SSH connection timeout, in milliseconds, to use with the builtin
>     sshfence fencer.
>   </description>
> </property>
> <!-- enable web HDFS -->
> <property>
>   <name>dfs.webhdfs.enabled</name>
>   <value>true</value>
> </property>
> </configuration>
>
>
>
> 2013-07-26 21:20:18,850 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage:
> nsid=291409768;bpid=BP-771660648-192.168.142.129-1374837820241;lv=-40;nsInfo=lv
> =-40;cid=CID-28365f0e-e4f1-45b0-a86a-bb37794b6672;nsid=291409768;c=0;bpid=BP-771660648-192.168.142.129-1374837820241
> 2013-07-26 21:20:18,870 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool
> BP-771660648-192.168.142.129-1374837820241 (storage id
> DS-713465905-192.168.142.131-5001
> 0-1374844418641) service to node1/192.168.142.129:8020 beginning handshake
> with NN
> 2013-07-26 21:20:18,873 FATAL
> org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for
> block pool Block pool BP-771660648-192.168.142.129-1374837820241 (storag
> e id DS-713465905-192.168.142.131-50010-1374844418641) service to
> node2/192.168.142.130:8020
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Invalid volume
> failure  config value: 3
>         at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.<init>(FsDatasetImpl.java:183)

Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB