Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Is DFSInputStream.read(long position,...) designed for multi threaded access?


Copy link to this message
-
Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
Andrew Purtell 2011-03-02, 22:38
I looked at HADOOP-6311 recently when deciding if I wanted to port the full FD-passing bits of HDFS-347 onto CDH3B3. I see Owen changed his -1 to -0, but that's not really comforting. How likely is this to go in? Would this go in to CDH if I posted a patch for HADOOP-6311 and HDFS-347?

My aim here is a fast local read path that respects security, obviously, something that I won't be supporting myself in production against an increasingly diverging upstream.

    - Andy
--- On Wed, 3/2/11, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> From: Todd Lipcon <[EMAIL PROTECTED]>
> Subject: Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
> To: "Jason Rutherglen" <[EMAIL PROTECTED]>
> Cc: [EMAIL PROTECTED]
> Date: Wednesday, March 2, 2011, 2:10 PM
> On Wed, Mar 2, 2011 at 8:17 AM, Jason
> Rutherglen
> <[EMAIL PROTECTED]>
> wrote:
> > Todd,
> >
> > Thanks for the reply.  I looked at HDFS-941 which seems to remove the
> > redundant creation of BlockReaders.  That and HDFS-347 will solve some
> > of the issues, however I think there's also the sendmsg() of the file
> > descriptor that also needs to be implemented?
>
> Yes, see HADOOP-6311. I have an up-to-date patch on this but have not
> yet gotten to posting it since we need 347 to be redone on trunk as
> well.
>
> -Todd