Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> hdfs du periodicity and hdfs not respond at that time


Copy link to this message
-
Re: hdfs du periodicity and hdfs not respond at that time
I missed the periodicity part of your question. Unfortunately the "du"
refresh interval is hard-coded today, although the "df" interval is
configurable. Perhaps this is a bug - I filed
https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.

Also, your problem reminded me of a similar issue my team and I faced
once before, and
http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
helped us temporarily there. Perhaps it may help you as well, its a
good tip.

On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <[EMAIL PROTECTED]> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <[EMAIL PROTECTED]>
>>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <[EMAIL PROTECTED]>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> > anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks &  Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB