Hi,

  I'm storing sequence files in the distributed cache which seems to be
stored somewher under each node's /tmp .../local/archive/ ... path.

In mapper code, I tried using SequenceFile.Reader with all possible
configurations (locally, distribtued) however, it can't find it. Are
sequence files supported in the distribued cache??

Thank you,
Mark

Hi Mark,

DistributedCache files, when accessed from a Task, exist on the local
file system. You should make sure the SequenceFile.Reader tries to
read it with a LocalFS than a HDFS instance.

On Thu, Oct 11, 2012 at 5:15 AM, Mark Olimpiati <[EMAIL PROTECTED]> wrote:
> Hi,
>
>   I'm storing sequence files in the distributed cache which seems to be
> stored somewher under each node's /tmp .../local/archive/ ... path.
>
> In mapper code, I tried using SequenceFile.Reader with all possible
> configurations (locally, distribtued) however, it can't find it. Are
> sequence files supported in the distribued cache??
>
> Thank you,
> Mark

--
Harsh J

Thanks for the reply Harsh, but as I said I tried locally too by using the
following:

     FileSystem localFs = cachedFiles[0].getFileSystem(new
Configuration());
Isn't the above supposed to give me the local file system ?? If yes, I
tried that but didn't work, if not, can you show one example?

Thank you,
Mark

On Wed, Oct 10, 2012 at 7:33 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi Mark,
>
> DistributedCache files, when accessed from a Task, exist on the local
> file system. You should make sure the SequenceFile.Reader tries to
> read it with a LocalFS than a HDFS instance.
>
> On Thu, Oct 11, 2012 at 5:15 AM, Mark Olimpiati <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> >   I'm storing sequence files in the distributed cache which seems to be
> > stored somewher under each node's /tmp .../local/archive/ ... path.
> >
> > In mapper code, I tried using SequenceFile.Reader with all possible
> > configurations (locally, distribtued) however, it can't find it. Are
> > sequence files supported in the distribued cache??
> >
> > Thank you,
> > Mark
>
>
>
> --
> Harsh J
>

On 11 October 2012 20:53, Mark Olimpiati <[EMAIL PROTECTED]> wrote:

> Thanks for the reply Harsh, but as I said I tried locally too by using the
> following:
>
>      FileSystem localFs = cachedFiles[0].getFileSystem(new
> Configuration());
>
>
> Isn't the above supposed to give me the local file system ?? If yes, I
> tried that but didn't work, if not, can you show one example?
>
> Thank you,
> Mark
gives you the default FS for that process, which can be different.

what about FileSystem.get("file://tmp",new Configuration())