Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> New Distributed Cache


+
Botelho, Andrew 2013-07-10, 17:15
+
Omkar Joshi 2013-07-10, 21:08
Copy link to this message
-
Re: New Distributed Cache
Also, once you have the array of URIs after calling getCacheFiles  you can
iterate over them using File class or Path (
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/Path.html#Path(java.net.URI)
)

Regards,
Shahab
On Wed, Jul 10, 2013 at 5:08 PM, Omkar Joshi <[EMAIL PROTECTED]> wrote:

> did you try JobContext.getCacheFiles() ?
>
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Wed, Jul 10, 2013 at 10:15 AM, Botelho, Andrew <[EMAIL PROTECTED]>wrote:
>
>> Hi,****
>>
>> ** **
>>
>> I am trying to store a file in the Distributed Cache during my Hadoop job.
>> ****
>>
>> In the driver class, I tell the job to store the file in the cache with
>> this code:****
>>
>> ** **
>>
>> Job job = Job.getInstance();****
>>
>> job.addCacheFile(new URI("file name"));****
>>
>> ** **
>>
>> That all compiles fine.  In the Mapper code, I try accessing the cached
>> file with this method:****
>>
>> ** **
>>
>> Path[] localPaths = context.getLocalCacheFiles();****
>>
>> ** **
>>
>> However, I am getting warnings that this method is deprecated.****
>>
>> Does anyone know the newest way to access cached files in the Mapper
>> code? (I am using Hadoop 2.0.5)****
>>
>> ** **
>>
>> Thanks in advance,****
>>
>> ** **
>>
>> Andrew****
>>
>
>
+
Omkar Joshi 2013-07-11, 18:08
+
Bill Q 2014-01-09, 16:10