|
|
-
Problem using distributed cachePeter Cogan 2012-12-06, 16:59
Hi ,
I want to use the distributed cache to allow my mappers to access data. In main, I'm using the command DistributedCache.addCacheFile(new URI("/user/peter/cacheFile/testCache1"), conf); Where /user/peter/cacheFile/testCache1 is a file that exists in hdfs Then, my setup function looks like this: public void setup(Context context) throws IOException, InterruptedException{ Configuration conf = context.getConfiguration(); Path[] localFiles = DistributedCache.getLocalCacheFiles(conf); //etc } However, this localFiles array is always null. I was initially running on a single-host cluster for testing, but I read that this will prevent the distributed cache from working. I tried with a pseudo-distributed, but that didn't work either I'm using hadoop 1.0.3 thanks Peter |