Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: hdfs du periodicity and hdfs not respond at that time


Copy link to this message
-
Re: hdfs du periodicity and hdfs not respond at that time
I'm using ext3 and use df instead of du is a good way to solve this
problem, thanks you all

2013/1/24 Harsh J <[EMAIL PROTECTED]>

> I missed the periodicity part of your question. Unfortunately the "du"
> refresh interval is hard-coded today, although the "df" interval is
> configurable. Perhaps this is a bug - I filed
> https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
>
> Also, your problem reminded me of a similar issue my team and I faced
> once before, and
>
> http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
> helped us temporarily there. Perhaps it may help you as well, its a
> good tip.
>
> On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <[EMAIL PROTECTED]> wrote:
> > Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> > dont't know how to solve this problem, 3 minutes not respond once an hour
> > is a big problem for me, any clue for this?
> >
> >
> > 2013/1/24 Harsh J <[EMAIL PROTECTED]>
> >>
> >> Hi,
> >>
> >> HDFS does this to estimate space reports. Perhaps the discussion here
> >> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
> >>
> >> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <[EMAIL PROTECTED]>
> >> wrote:
> >> > hi all,
> >> > I found hdfs du periodicity(one hour), and because my disk is big, the
> >> > smallest one is 15T, so when hdfs exec du, datanode will not respond
> for
> >> > about 3 minuts because of io loading, this cause a lot of problem,
> >> > anybody
> >> > knows why hdfs doing this and how to disable it?
> >> >
> >> > --
> >> > Thanks &  Regards
> >> > Xibin Liu
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
> >
> >
> >
> >
> > --
> > Thanks & Best Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>

--
Thanks & Best Regards
Xibin Liu
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB