Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: DataNode heartbeat average time peaks


Copy link to this message
-
Re: DataNode heartbeat average time peaks
Worst case if it puts too much strain: randomly failing clients and
missing DN heartbeats leading to unnecessary dead node appearances.
Worth using a release with HDFS-1457 in or patching your NN/SNN for
that - helps shape those graphs.

On Mon, Mar 25, 2013 at 9:59 PM, Ivan Tretyakov
<[EMAIL PROTECTED]> wrote:
> Thanks Harsh!
>
> My image size is about 3.1 Gb.
> Yes, I think feature from HDFS-1457 is what I need, but unfortunately it is
> not available in version of hadoop we use.
>
> What kind of risks pose by these peaks.
>
>
> On Mon, Mar 25, 2013 at 7:31 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> What's your fsimage size? If its too high you would want to control
>> the checkpoint transfer bandwidth to not affect the load at the NN.
>> This is available via the JIRA HDFS-1457.
>>
>>
>> On Mon, Mar 25, 2013 at 8:13 PM, Ivan Tretyakov
>> <[EMAIL PROTECTED]> wrote:
>> > Hi!
>> >
>> > We see DataNode heartbeat average time peaks in Ganglia up to 20-70
>> > seconds
>> > while SecondaryNameNode performs checkpointing.
>> > See attached screenshots please.
>> >
>> > I would like to clarify if it is Ok, or not. And what kind of
>> > consequences
>> > and risks it could bring up.
>> >
>> > --
>> > Best Regards
>> > Ivan Tretyakov
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Best Regards
> Ivan Tretyakov
>

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB