Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> datanode write forwarding


+
Jay Vyas 2012-12-18, 20:56
+
Mohammad Tariq 2012-12-18, 21:07
+
Harsh J 2012-12-18, 22:13
+
Chris Embree 2012-12-18, 22:19
Copy link to this message
-
Re: datanode write forwarding
Can't speak for their explanation (not sure what doc you seem to
be referring to?) but the behavior of mirror-first-disk-next has been
there since 0.20 at least - so yes, 1.0 too has the same behavior.

If you speak of the doc
http://hadoop.apache.org/docs/stable/hdfs_design.html#Replication+Pipelining,
that seems to be a bit outdated overall. We could change the text -
mind filing a JIRA and doing so?

On Wed, Dec 19, 2012 at 3:49 AM, Chris Embree <[EMAIL PROTECTED]> wrote:
> Harsh,
>
> Is that a change from 1.0 code?  Hortonworks explains it a little
> differently.
>
> Thanks for the details and pointer to the code.
>
> Chris (another one)
>
> On Dec 18, 2012 5:14 PM, "Harsh J" <[EMAIL PROTECTED]> wrote:
>>
>> Hi,
>>
>> The received write packet is directly socket-written to the next
>> node's receiver (async process - we don't wait for it to hit the disk
>> there), and then written onto the disk of self().
>>
>> You may see HDFS classes BlockReceiver and PacketReceiver in trunk for
>> more information.
>>
>> On Wed, Dec 19, 2012 at 2:26 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>> > Hi :
>> >
>> > Do data nodes, when they receive write requests, forward the request to
>> > the
>> > next datanode in the chain after appending to a file? Or rather, do the
>> > write and forward asynchronously and at the same time?
>> >
>> > --
>> > Jay Vyas
>> > http://jayunit100.blogspot.com
>>
>>
>>
>> --
>> Harsh J

--
Harsh J
+
Chris Embree 2012-12-18, 21:09
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB