Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Is DFSInputStream.read(long position,...) designed for multi threaded access?


+
Jason Rutherglen 2011-03-02, 00:24
+
Todd Lipcon 2011-03-02, 01:58
Copy link to this message
-
Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
Todd,

Thanks for the reply.  I looked at HDFS-941 which seems to remove the
redundant creation of BlockReaders.  That and HDFS-347 will solve some
of the issues, however I think there's also the sendmsg() of the file
descriptor that also needs to be implemented?

Jason

On Tue, Mar 1, 2011 at 5:58 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> Hi Jason,
>
> Yes, this method is currently very inefficient.. HDFS-941 will
> hopefully improve this situation, but currently there's no
> particularly efficient way to do multithreaded random access.
>
> -Todd
>
> On Tue, Mar 1, 2011 at 4:24 PM, Jason Rutherglen
> <[EMAIL PROTECTED]> wrote:
>> It's unsynchronized however it's creating a new BlockReader on each
>> call, that seems like a problem?
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
+
Todd Lipcon 2011-03-02, 22:10
+
Andrew Purtell 2011-03-02, 22:38
+
Todd Lipcon 2011-03-02, 22:42
+
Jason Rutherglen 2011-03-02, 23:48
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB