Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> HDFS: buffer before contacts Namenode?

Copy link to this message
Re: HDFS: buffer before contacts Namenode?
DataNode only buffers a packet before it contacts NameNode for allocating
DataNodes to place the block. The doc you read might be too old.


On 8/9/10 7:14 PM, "elton sky" <[EMAIL PROTECTED]> wrote:

> hello folks,
> I can see from the design doc of HDFS, says: client will buffer a block size
> worth of data before contacting namenode for data node info. This is a network
> throughput optimal way.
> However, I could not find this buffer processing procedure in source code.
> In DFSClient.DataStreamer, it waits for dataqueue to be not empty and starts
> to request namenode and build a pipeline. The number of packets in the
> dataqueue is always 1 when this happens! 
> I am confused here. Can anyone address this if I am wrong?