Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> New Distributed Cache


+
Botelho, Andrew 2013-07-10, 17:15
+
Omkar Joshi 2013-07-10, 21:08
Copy link to this message
-
Re: New Distributed Cache
Also, once you have the array of URIs after calling getCacheFiles  you can
iterate over them using File class or Path (
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/Path.html#Path(java.net.URI)
)

Regards,
Shahab
On Wed, Jul 10, 2013 at 5:08 PM, Omkar Joshi <[EMAIL PROTECTED]> wrote:

> did you try JobContext.getCacheFiles() ?
>
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Wed, Jul 10, 2013 at 10:15 AM, Botelho, Andrew <[EMAIL PROTECTED]>wrote:
>
>> Hi,****
>>
>> ** **
>>
>> I am trying to store a file in the Distributed Cache during my Hadoop job.
>> ****
>>
>> In the driver class, I tell the job to store the file in the cache with
>> this code:****
>>
>> ** **
>>
>> Job job = Job.getInstance();****
>>
>> job.addCacheFile(new URI("file name"));****
>>
>> ** **
>>
>> That all compiles fine.  In the Mapper code, I try accessing the cached
>> file with this method:****
>>
>> ** **
>>
>> Path[] localPaths = context.getLocalCacheFiles();****
>>
>> ** **
>>
>> However, I am getting warnings that this method is deprecated.****
>>
>> Does anyone know the newest way to access cached files in the Mapper
>> code? (I am using Hadoop 2.0.5)****
>>
>> ** **
>>
>> Thanks in advance,****
>>
>> ** **
>>
>> Andrew****
>>
>
>
+
Omkar Joshi 2013-07-11, 18:08
+
Bill Q 2014-01-09, 16:10
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB