-Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
Andrew Purtell 2011-03-02, 22:38
I looked at HADOOP-6311 recently when deciding if I wanted to port the full FD-passing bits of HDFS-347 onto CDH3B3. I see Owen changed his -1 to -0, but that's not really comforting. How likely is this to go in? Would this go in to CDH if I posted a patch for HADOOP-6311 and HDFS-347?
My aim here is a fast local read path that respects security, obviously, something that I won't be supporting myself in production against an increasingly diverging upstream.
--- On Wed, 3/2/11, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> From: Todd Lipcon <[EMAIL PROTECTED]>
> Subject: Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
> To: "Jason Rutherglen" <[EMAIL PROTECTED]>
> Cc: [EMAIL PROTECTED]
> Date: Wednesday, March 2, 2011, 2:10 PM
> On Wed, Mar 2, 2011 at 8:17 AM, Jason
> <[EMAIL PROTECTED]>
> > Todd,
> > Thanks for the reply. I looked at HDFS-941 which seems to remove the
> > redundant creation of BlockReaders. That and HDFS-347 will solve some
> > of the issues, however I think there's also the sendmsg() of the file
> > descriptor that also needs to be implemented?
> Yes, see HADOOP-6311. I have an up-to-date patch on this but have not
> yet gotten to posting it since we need 347 to be redone on trunk as