Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Is DFSInputStream.read(long position,...) designed for multi threaded access?


Copy link to this message
-
Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
I looked at HADOOP-6311 recently when deciding if I wanted to port the full FD-passing bits of HDFS-347 onto CDH3B3. I see Owen changed his -1 to -0, but that's not really comforting. How likely is this to go in? Would this go in to CDH if I posted a patch for HADOOP-6311 and HDFS-347?

My aim here is a fast local read path that respects security, obviously, something that I won't be supporting myself in production against an increasingly diverging upstream.

    - Andy
--- On Wed, 3/2/11, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> From: Todd Lipcon <[EMAIL PROTECTED]>
> Subject: Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
> To: "Jason Rutherglen" <[EMAIL PROTECTED]>
> Cc: [EMAIL PROTECTED]
> Date: Wednesday, March 2, 2011, 2:10 PM
> On Wed, Mar 2, 2011 at 8:17 AM, Jason
> Rutherglen
> <[EMAIL PROTECTED]>
> wrote:
> > Todd,
> >
> > Thanks for the reply.  I looked at HDFS-941 which seems to remove the
> > redundant creation of BlockReaders.  That and HDFS-347 will solve some
> > of the issues, however I think there's also the sendmsg() of the file
> > descriptor that also needs to be implemented?
>
> Yes, see HADOOP-6311. I have an up-to-date patch on this but have not
> yet gotten to posting it since we need 347 to be redone on trunk as
> well.
>
> -Todd

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB