Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Problem using distributed cache


Copy link to this message
-
Re: Problem using distributed cache
Hello Peter
In my, humble, experience I never get hadoop 1.0.3 to work with
distributed cache and the new api (mapreduce). with the old api it works.
giovanni

P.S. I already tried the approaches suggested by both Dhaval and  Harsh J
On 12/06/2012 05:59 PM, Peter Cogan wrote:
>
> Hi ,
>
> I want to use the distributed cache to allow my mappers to access
> data. In main, I'm using the command
>
> |DistributedCache.addCacheFile(new URI("/user/peter/cacheFile/testCache1"), conf);
> |
>
> Where /user/peter/cacheFile/testCache1 is a file that exists in hdfs
>
> Then, my setup function looks like this:
>
> |public void setup(Context context) throws IOException, InterruptedException{
>     Configuration conf = context.getConfiguration();
>     Path[] localFiles = DistributedCache.getLocalCacheFiles(conf);
>     //etc
> }
> |
>
> However, this localFiles array is always null.
>
> I was initially running on a single-host cluster for testing, but I
> read that this will prevent the distributed cache from working. I
> tried with a pseudo-distributed, but that didn't work either
>
> I'm using hadoop 1.0.3
>
> thanks Peter
>
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB