|
|
-
Re: hdfs du periodicity and hdfs not respond at that timeXibin Liu 2013-01-25, 02:34
I'm using ext3 and use df instead of du is a good way to solve this
problem, thanks you all 2013/1/24 Harsh J <[EMAIL PROTECTED]> > I missed the periodicity part of your question. Unfortunately the "du" > refresh interval is hard-coded today, although the "df" interval is > configurable. Perhaps this is a bug - I filed > https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable. > > Also, your problem reminded me of a similar issue my team and I faced > once before, and > > http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/ > helped us temporarily there. Perhaps it may help you as well, its a > good tip. > > On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <[EMAIL PROTECTED]> wrote: > > Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still > > dont't know how to solve this problem, 3 minutes not respond once an hour > > is a big problem for me, any clue for this? > > > > > > 2013/1/24 Harsh J <[EMAIL PROTECTED]> > >> > >> Hi, > >> > >> HDFS does this to estimate space reports. Perhaps the discussion here > >> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2 > >> > >> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <[EMAIL PROTECTED]> > >> wrote: > >> > hi all, > >> > I found hdfs du periodicity(one hour), and because my disk is big, the > >> > smallest one is 15T, so when hdfs exec du, datanode will not respond > for > >> > about 3 minuts because of io loading, this cause a lot of problem, > >> > anybody > >> > knows why hdfs doing this and how to disable it? > >> > > >> > -- > >> > Thanks & Regards > >> > Xibin Liu > >> > > >> > >> > >> > >> -- > >> Harsh J > > > > > > > > > > -- > > Thanks & Best Regards > > Xibin Liu > > > > > > -- > Harsh J > -- Thanks & Best Regards Xibin Liu |