Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Reading Sequence File from Hadoop Distributed Cache ..


Copy link to this message
-
Re: Reading Sequence File from Hadoop Distributed Cache ..
Thanks for the reply Harsh, but as I said I tried locally too by using the
following:

     FileSystem localFs = cachedFiles[0].getFileSystem(new
Configuration());
Isn't the above supposed to give me the local file system ?? If yes, I
tried that but didn't work, if not, can you show one example?

Thank you,
Mark

On Wed, Oct 10, 2012 at 7:33 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi Mark,
>
> DistributedCache files, when accessed from a Task, exist on the local
> file system. You should make sure the SequenceFile.Reader tries to
> read it with a LocalFS than a HDFS instance.
>
> On Thu, Oct 11, 2012 at 5:15 AM, Mark Olimpiati <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> >   I'm storing sequence files in the distributed cache which seems to be
> > stored somewher under each node's /tmp .../local/archive/ ... path.
> >
> > In mapper code, I tried using SequenceFile.Reader with all possible
> > configurations (locally, distribtued) however, it can't find it. Are
> > sequence files supported in the distribued cache??
> >
> > Thank you,
> > Mark
>
>
>
> --
> Harsh J
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB