Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Problem using distributed cache


Copy link to this message
-
Problem using distributed cache
Hi ,

I want to use the distributed cache to allow my mappers to access data. In
main, I'm using the command

DistributedCache.addCacheFile(new
URI("/user/peter/cacheFile/testCache1"), conf);

Where /user/peter/cacheFile/testCache1 is a file that exists in hdfs

Then, my setup function looks like this:

public void setup(Context context) throws IOException, InterruptedException{
    Configuration conf = context.getConfiguration();
    Path[] localFiles = DistributedCache.getLocalCacheFiles(conf);
    //etc
}

However, this localFiles array is always null.

I was initially running on a single-host cluster for testing, but I read
that this will prevent the distributed cache from working. I tried with a
pseudo-distributed, but that didn't work either

I'm using hadoop 1.0.3

thanks Peter