Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Is DFSInputStream.read(long position,...) designed for multi threaded access?


+
Jason Rutherglen 2011-03-02, 00:24
+
Todd Lipcon 2011-03-02, 01:58
+
Jason Rutherglen 2011-03-02, 16:17
+
Todd Lipcon 2011-03-02, 22:10
+
Andrew Purtell 2011-03-02, 22:38
+
Todd Lipcon 2011-03-02, 22:42
Copy link to this message
-
Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
I wonder how much of the Android code would be used?  Eg, are we
passing data over domain sockets or are we only planning on using
sendmsg()?

On Wed, Mar 2, 2011 at 2:38 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> I looked at HADOOP-6311 recently when deciding if I wanted to port the full FD-passing bits of HDFS-347 onto CDH3B3. I see Owen changed his -1 to -0, but that's not really comforting. How likely is this to go in? Would this go in to CDH if I posted a patch for HADOOP-6311 and HDFS-347?
>
> My aim here is a fast local read path that respects security, obviously, something that I won't be supporting myself in production against an increasingly diverging upstream.
>
>    - Andy
>
>
> --- On Wed, 3/2/11, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>
>> From: Todd Lipcon <[EMAIL PROTECTED]>
>> Subject: Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
>> To: "Jason Rutherglen" <[EMAIL PROTECTED]>
>> Cc: [EMAIL PROTECTED]
>> Date: Wednesday, March 2, 2011, 2:10 PM
>> On Wed, Mar 2, 2011 at 8:17 AM, Jason
>> Rutherglen
>> <[EMAIL PROTECTED]>
>> wrote:
>> > Todd,
>> >
>> > Thanks for the reply.  I looked at HDFS-941 which seems to remove the
>> > redundant creation of BlockReaders.  That and HDFS-347 will solve some
>> > of the issues, however I think there's also the sendmsg() of the file
>> > descriptor that also needs to be implemented?
>>
>> Yes, see HADOOP-6311. I have an up-to-date patch on this but have not
>> yet gotten to posting it since we need 347 to be redone on trunk as
>> well.
>>
>> -Todd
>
>
>
>
>