Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> message transmission in Hadoop


+
Da Zheng 2011-01-29, 03:08
Copy link to this message
-
Re: message transmission in Hadoop
Hey Da,

You may have observed https://issues.apache.org/jira/browse/HDFS-1601.

Regards,
Jeff

On Fri, Jan 28, 2011 at 7:08 PM, Da Zheng <[EMAIL PROTECTED]> wrote:

> Hello,
>
> I monitored system calls of HDFS with systemtap and found HDFS actually
> sends
> many 1-byte data to the network. I could also see many 8-byte and 64-byte
> data
> written to the OS though I don't know whether they are written to the disk
> or
> sent to the network. I did see many 8-byte data sent to the network. The
> number
> of these data is several times more than 64KB data packet sent by HDFS.
>
> Could anyone tell me why HDFS sends so many small packets? heartbeat
> messages?
> RPCs? It doesn't seem to me these messages can be just 1 byte.
>
> Thanks,
> Da
>
+
Da Zheng 2011-01-31, 00:56
+
Da Zheng 2011-01-31, 16:59
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB