Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> hdfs du periodicity and hdfs not respond at that time


+
Xibin Liu 2013-01-24, 07:21
+
Harsh J 2013-01-24, 07:41
+
Xibin Liu 2013-01-24, 07:54
Copy link to this message
-
Re: hdfs du periodicity and hdfs not respond at that time
I missed the periodicity part of your question. Unfortunately the "du"
refresh interval is hard-coded today, although the "df" interval is
configurable. Perhaps this is a bug - I filed
https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.

Also, your problem reminded me of a similar issue my team and I faced
once before, and
http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
helped us temporarily there. Perhaps it may help you as well, its a
good tip.

On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <[EMAIL PROTECTED]> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <[EMAIL PROTECTED]>
>>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <[EMAIL PROTECTED]>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> > anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks &  Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>

--
Harsh J
+
Chris Embree 2013-01-24, 12:56