Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Is DFSInputStream.read(long position,...) designed for multi threaded access?


+
Jason Rutherglen 2011-03-02, 00:24
+
Todd Lipcon 2011-03-02, 01:58
Copy link to this message
-
Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
Jason Rutherglen 2011-03-02, 16:17
Todd,

Thanks for the reply.  I looked at HDFS-941 which seems to remove the
redundant creation of BlockReaders.  That and HDFS-347 will solve some
of the issues, however I think there's also the sendmsg() of the file
descriptor that also needs to be implemented?

Jason

On Tue, Mar 1, 2011 at 5:58 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> Hi Jason,
>
> Yes, this method is currently very inefficient.. HDFS-941 will
> hopefully improve this situation, but currently there's no
> particularly efficient way to do multithreaded random access.
>
> -Todd
>
> On Tue, Mar 1, 2011 at 4:24 PM, Jason Rutherglen
> <[EMAIL PROTECTED]> wrote:
>> It's unsynchronized however it's creating a new BlockReader on each
>> call, that seems like a problem?
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
+
Todd Lipcon 2011-03-02, 22:10
+
Andrew Purtell 2011-03-02, 22:38
+
Todd Lipcon 2011-03-02, 22:42
+
Jason Rutherglen 2011-03-02, 23:48