Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Cache file conflict


Copy link to this message
-
Re: Cache file conflict
Public Network Services 2013-08-30, 03:49
It does seem relevant, but what is the workaround?

The JAR causing the issue is the same file all the time.
On Thu, Aug 29, 2013 at 5:23 PM, Omkar Joshi <[EMAIL PROTECTED]> wrote:

> you should check this
> https://issues.apache.org/jira/browse/MAPREDUCE-4493?focusedCommentId=13713706&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13713706
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Thu, Aug 29, 2013 at 5:06 PM, Public Network Services <
> [EMAIL PROTECTED]> wrote:
>
>> Hi...
>>
>> After updating the source JARs of an application that launches a second
>> job while running a MR job, the following error keeps occurring:
>>
>> org.apache.hadoop.mapred.InvalidJobConfException: cache file
>> (mapreduce.job.cache.files) scheme: "hdfs", host: "server", port: 9000,
>> file:
>> "/tmp/hadoop-yarn/staging/root/.staging/job_1367474197612_0887/libjars/Some.jar",
>> conflicts with cache file (mapreduce.job.cache.files)
>> hdfs://server:9000/tmp/hadoop-yarn/staging/root/.staging/job_1367474197612_0888/libjars/Some.jar
>>     at
>> org.apache.hadoop.mapreduce.v2.util.MRApps.parseDistributedCacheArtifacts(MRApps.java:338)
>>     at
>> org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:273)
>>     at
>> org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:419)
>>     at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:288)
>>     at
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
>>     at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
>>     at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at javax.security.auth.Subject.doAs(Subject.java:396)
>>     at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
>>     at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
>> ............
>>
>> where job_1367474197612_0887 is the name of the initial job,
>> job_1367474197612_0888 is the name of the subsequent job, and Some.jar is a
>> JAR file specific to the application.
>>
>> Any ideas as to how the above error could be eliminated?
>>
>> Thanks!
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.