Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Reading Sequence File from Hadoop Distributed Cache ..


Copy link to this message
-
Re: Reading Sequence File from Hadoop Distributed Cache ..
Thanks for the reply Harsh, but as I said I tried locally too by using the
following:

     FileSystem localFs = cachedFiles[0].getFileSystem(new
Configuration());
Isn't the above supposed to give me the local file system ?? If yes, I
tried that but didn't work, if not, can you show one example?

Thank you,
Mark

On Wed, Oct 10, 2012 at 7:33 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi Mark,
>
> DistributedCache files, when accessed from a Task, exist on the local
> file system. You should make sure the SequenceFile.Reader tries to
> read it with a LocalFS than a HDFS instance.
>
> On Thu, Oct 11, 2012 at 5:15 AM, Mark Olimpiati <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> >   I'm storing sequence files in the distributed cache which seems to be
> > stored somewher under each node's /tmp .../local/archive/ ... path.
> >
> > In mapper code, I tried using SequenceFile.Reader with all possible
> > configurations (locally, distribtued) however, it can't find it. Are
> > sequence files supported in the distribued cache??
> >
> > Thank you,
> > Mark
>
>
>
> --
> Harsh J
>