Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> datanode write forwarding


Copy link to this message
-
Re: datanode write forwarding
Can't speak for their explanation (not sure what doc you seem to
be referring to?) but the behavior of mirror-first-disk-next has been
there since 0.20 at least - so yes, 1.0 too has the same behavior.

If you speak of the doc
http://hadoop.apache.org/docs/stable/hdfs_design.html#Replication+Pipelining,
that seems to be a bit outdated overall. We could change the text -
mind filing a JIRA and doing so?

On Wed, Dec 19, 2012 at 3:49 AM, Chris Embree <[EMAIL PROTECTED]> wrote:
> Harsh,
>
> Is that a change from 1.0 code?  Hortonworks explains it a little
> differently.
>
> Thanks for the details and pointer to the code.
>
> Chris (another one)
>
> On Dec 18, 2012 5:14 PM, "Harsh J" <[EMAIL PROTECTED]> wrote:
>>
>> Hi,
>>
>> The received write packet is directly socket-written to the next
>> node's receiver (async process - we don't wait for it to hit the disk
>> there), and then written onto the disk of self().
>>
>> You may see HDFS classes BlockReceiver and PacketReceiver in trunk for
>> more information.
>>
>> On Wed, Dec 19, 2012 at 2:26 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>> > Hi :
>> >
>> > Do data nodes, when they receive write requests, forward the request to
>> > the
>> > next datanode in the chain after appending to a file? Or rather, do the
>> > write and forward asynchronously and at the same time?
>> >
>> > --
>> > Jay Vyas
>> > http://jayunit100.blogspot.com
>>
>>
>>
>> --
>> Harsh J

--
Harsh J