Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: DataNode heartbeat average time peaks


Copy link to this message
-
Re: DataNode heartbeat average time peaks
Harsh J 2013-03-25, 16:32
Worst case if it puts too much strain: randomly failing clients and
missing DN heartbeats leading to unnecessary dead node appearances.
Worth using a release with HDFS-1457 in or patching your NN/SNN for
that - helps shape those graphs.

On Mon, Mar 25, 2013 at 9:59 PM, Ivan Tretyakov
<[EMAIL PROTECTED]> wrote:
> Thanks Harsh!
>
> My image size is about 3.1 Gb.
> Yes, I think feature from HDFS-1457 is what I need, but unfortunately it is
> not available in version of hadoop we use.
>
> What kind of risks pose by these peaks.
>
>
> On Mon, Mar 25, 2013 at 7:31 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> What's your fsimage size? If its too high you would want to control
>> the checkpoint transfer bandwidth to not affect the load at the NN.
>> This is available via the JIRA HDFS-1457.
>>
>>
>> On Mon, Mar 25, 2013 at 8:13 PM, Ivan Tretyakov
>> <[EMAIL PROTECTED]> wrote:
>> > Hi!
>> >
>> > We see DataNode heartbeat average time peaks in Ganglia up to 20-70
>> > seconds
>> > while SecondaryNameNode performs checkpointing.
>> > See attached screenshots please.
>> >
>> > I would like to clarify if it is Ok, or not. And what kind of
>> > consequences
>> > and risks it could bring up.
>> >
>> > --
>> > Best Regards
>> > Ivan Tretyakov
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Best Regards
> Ivan Tretyakov
>

--
Harsh J