Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> org.apache.hadoop.io.MapFile.Reader.Reader(FileSystem fs, String dirName, Configuration conf) constructor ignores the FileSystem


Copy link to this message
-
Re: org.apache.hadoop.io.MapFile.Reader.Reader(FileSystem fs, String dirName, Configuration conf) constructor ignores the FileSystem
Thanks Harsh…..  :-)

Cheers,
Subroto Sanyal

On May 18, 2012, at 4:30 PM, Harsh J wrote:

> With 0.23/2.x onwards, to have better behavior, its your Path that
> will determine the FS in SequenceFile readers/writers. Hence, always
> make sure your Path carries the proper URI if it is supposed to belong
> to a different FS. You can look at one of the many Path constructors
> to help you get such an object.
>
> The change was mainly done to counter situations where a path object
> belongs to a different FS (has a URI), and a FileSystem object passed
> along belongs to yet another. To avoid this form of confusion, it was
> decided to always evaluate the FS out of the Path itself.
>
> P.s. (Tip) Use the new Builder API for SequenceFile reader/writer in
> 0.23/2.x, they are nicer to have.
>
> P.p.s. Also, if your question is CDH specific, please always use the
> [EMAIL PROTECTED] lists instead of the Apache ones. You can
> subscribe to/browse it via
> https://groups.google.com/a/cloudera.org/forum/#!forum/cdh-user.
>
> On Fri, May 18, 2012 at 7:43 PM, Subroto <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> The constructor of Reader class ignores the FileSystem parameter provided in the constructor parameter.
>> This results in creation of Path on the basis of default FileSystem mentioned in the configuration even though user wants to pass LocalFileSystem.
>>
>> The behavior has been verified with cdh4b2 version. Is this a correct behavior of the API?
>>
>> Cheers,
>> Subroto Sanyal
>
>
>
> --
> Harsh J