Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: hdfs du periodicity and hdfs not respond at that time


Copy link to this message
-
Re: hdfs du periodicity and hdfs not respond at that time
I'm using ext3 and use df instead of du is a good way to solve this
problem, thanks you all

2013/1/24 Harsh J <[EMAIL PROTECTED]>

> I missed the periodicity part of your question. Unfortunately the "du"
> refresh interval is hard-coded today, although the "df" interval is
> configurable. Perhaps this is a bug - I filed
> https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
>
> Also, your problem reminded me of a similar issue my team and I faced
> once before, and
>
> http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
> helped us temporarily there. Perhaps it may help you as well, its a
> good tip.
>
> On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <[EMAIL PROTECTED]> wrote:
> > Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> > dont't know how to solve this problem, 3 minutes not respond once an hour
> > is a big problem for me, any clue for this?
> >
> >
> > 2013/1/24 Harsh J <[EMAIL PROTECTED]>
> >>
> >> Hi,
> >>
> >> HDFS does this to estimate space reports. Perhaps the discussion here
> >> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
> >>
> >> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <[EMAIL PROTECTED]>
> >> wrote:
> >> > hi all,
> >> > I found hdfs du periodicity(one hour), and because my disk is big, the
> >> > smallest one is 15T, so when hdfs exec du, datanode will not respond
> for
> >> > about 3 minuts because of io loading, this cause a lot of problem,
> >> > anybody
> >> > knows why hdfs doing this and how to disable it?
> >> >
> >> > --
> >> > Thanks &  Regards
> >> > Xibin Liu
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
> >
> >
> >
> >
> > --
> > Thanks & Best Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>

--
Thanks & Best Regards
Xibin Liu