Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> why DFSClient.BlockReader doesnot support seek(long) can I use skip() instead?


Copy link to this message
-
Re: why DFSClient.BlockReader doesnot support seek(long) can I use skip() instead?
thanks, man,
these small files will be frequently accessed, our idea is that by
caching data location from namenode, we may relieve  the load of
NameNode and maybe also save sometime at network communication.
On Tue, Sep 21, 2010 at 1:10 AM, Allen Wittenauer
<[EMAIL PROTECTED]> wrote:
>
> On Sep 19, 2010, at 7:57 PM, steven zhuang wrote:
>
>> hi, all,
>>       I have sent this mail in common user list before, duplicate it
>> here to seek for more help from experts.
>
> You'll likely have more luck on hdfs-dev.
>
>>       I am wondering why seek(long) is disabled in HDFS.BlockReader?
>> Can I use skip(long) to replace this seek(long)?
>>
>>       I have a bunch of small files, each is less than a block in size.
>
>        In other words, using Hadoop against recommended best practices. :)
>
>
>> In
>> my program, given the file/block information, I will try to start a
>> process on each datanode and try to read from the HDFS directly
>> through a socket connection to the datanode.
>
>        Again, using Hadoop against best practices.
>
>>      The read requires seek OP on the file, cause the file I used is
>> TFile, which requires the underlying class to be seekable.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB