Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> local job directory customization


Copy link to this message
-
Re: local job directory customization
https://issues.apache.org/jira/browse/MAPREDUCE-2072 has been logged.

On Mon, Sep 13, 2010 at 3:43 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Hi,
> I saw the following in our unit test log on build machine:
>
> Caused by: org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/tmp/hadoop-build/mapred/system/job_local_0002/job.split': No such file or directory
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:195)
> at org.apache.hadoop.util.Shell.run(Shell.java:134)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
>
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
> at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:492)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:484)
>
> at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:286)
> at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:373)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
>
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:507)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:414)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:406)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:285)
>
> at org.apache.hadoop.mapred.JobClient.writeSplitsFileHeader(JobClient.java:1057)
> at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:873)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:822)
>
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
> at org.apache.hadoop.mapred.HadoopClient.runJob(HadoopClient.java:177)
>
> I guess the reason was that multiple unit tests (from different build
> projects) interfere with each other.
> Is there a way to customize local job directory ?
>
> Thanks
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB