Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> What if one of the directory(dfs.name.dir) rw error ?


+
ccxixicc 2011-05-25, 06:38
+
Harsh J 2011-05-25, 07:19
+
Tom Hall 2011-05-25, 10:32
+
ccxixicc 2011-05-25, 08:07
Copy link to this message
-
Re: What if one of the directory(dfs.name.dir) rw error ?
>>I dont know how to simulate a disk failure..

Couple of things you could do. chmod 000 is one.
1. umount -l
2. mount ro only
3. If machine has hot swappable disks, pull out a disk.

-Bharath

________________________________
From: ccxixicc <[EMAIL PROTECTED]>
To: hdfs-user <[EMAIL PROTECTED]>
Sent: Wednesday, May 25, 2011 1:07 AM
Subject: Re: What if one of the directory(dfs.name.dir) rw error ?
I'm using 0.20.2. 

I had some test. I dont know how to simulate a disk failure, just chmod 000 dir1, the namenode shutdown immediately. And NN will hang if the nfs server down.

 
 
------------------ Original ------------------
From:  "Harsh J"<[EMAIL PROTECTED]>;
Date:  Wed, May 25, 2011 03:49 PM
To:  "hdfs-user"<[EMAIL PROTECTED]>;
Subject:  Re: What if one of the directory(dfs.name.dir) rw error ?
 
Yes. But depending on the version you're using, you may have to
manually restart the NN after fixing the mount points, to get the
directories in action again.

2011/5/25 ccxixicc <[EMAIL PROTECTED]>:
>
> Hi,all
> I set dfs.name.dir to a comma-delimited list of directories, dir1 is in
> /dev/sdb1 dir2 is in /dev/sdb2 and dir3 is nfs derectory.
> What happens if /dev/sdb1 disk error, so dir1 cannot be read and write?
> What happens if nfs server down, so dir3 cannot be read and write?
> Will hadoop ignore the bad directory and use the good directory and continue
> server?
> Thanks.
>

--
Harsh J
+
Thanh Do 2011-05-26, 03:48
+
Konstantin Boudnik 2011-05-26, 04:42
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB