Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Problem using distributed cache


Copy link to this message
-
Re: Problem using distributed cache
Hello Peter
In my, humble, experience I never get hadoop 1.0.3 to work with
distributed cache and the new api (mapreduce). with the old api it works.
giovanni

P.S. I already tried the approaches suggested by both Dhaval and  Harsh J
On 12/06/2012 05:59 PM, Peter Cogan wrote:
>
> Hi ,
>
> I want to use the distributed cache to allow my mappers to access
> data. In main, I'm using the command
>
> |DistributedCache.addCacheFile(new URI("/user/peter/cacheFile/testCache1"), conf);
> |
>
> Where /user/peter/cacheFile/testCache1 is a file that exists in hdfs
>
> Then, my setup function looks like this:
>
> |public void setup(Context context) throws IOException, InterruptedException{
>     Configuration conf = context.getConfiguration();
>     Path[] localFiles = DistributedCache.getLocalCacheFiles(conf);
>     //etc
> }
> |
>
> However, this localFiles array is always null.
>
> I was initially running on a single-host cluster for testing, but I
> read that this will prevent the distributed cache from working. I
> tried with a pseudo-distributed, but that didn't work either
>
> I'm using hadoop 1.0.3
>
> thanks Peter
>
>