Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> message transmission in Hadoop

Da Zheng 2011-01-29, 03:08
Copy link to this message
Re: message transmission in Hadoop
Hey Da,

You may have observed https://issues.apache.org/jira/browse/HDFS-1601.


On Fri, Jan 28, 2011 at 7:08 PM, Da Zheng <[EMAIL PROTECTED]> wrote:

> Hello,
> I monitored system calls of HDFS with systemtap and found HDFS actually
> sends
> many 1-byte data to the network. I could also see many 8-byte and 64-byte
> data
> written to the OS though I don't know whether they are written to the disk
> or
> sent to the network. I did see many 8-byte data sent to the network. The
> number
> of these data is several times more than 64KB data packet sent by HDFS.
> Could anyone tell me why HDFS sends so many small packets? heartbeat
> messages?
> RPCs? It doesn't seem to me these messages can be just 1 byte.
> Thanks,
> Da
Da Zheng 2011-01-31, 00:56
Da Zheng 2011-01-31, 16:59