Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> local job directory customization


Copy link to this message
-
Re: local job directory customization
https://issues.apache.org/jira/browse/MAPREDUCE-2072 has been logged.

On Mon, Sep 13, 2010 at 3:43 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Hi,
> I saw the following in our unit test log on build machine:
>
> Caused by: org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/tmp/hadoop-build/mapred/system/job_local_0002/job.split': No such file or directory
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:195)
> at org.apache.hadoop.util.Shell.run(Shell.java:134)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
>
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
> at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:492)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:484)
>
> at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:286)
> at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:373)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
>
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:507)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:414)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:406)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:285)
>
> at org.apache.hadoop.mapred.JobClient.writeSplitsFileHeader(JobClient.java:1057)
> at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:873)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:822)
>
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
> at org.apache.hadoop.mapred.HadoopClient.runJob(HadoopClient.java:177)
>
> I guess the reason was that multiple unit tests (from different build
> projects) interfere with each other.
> Is there a way to customize local job directory ?
>
> Thanks
>