|
|
-
Fwd: Job Jar file does not existJieru Shi 2012-11-24, 03:19
Hi
I'm using embeded Pig to implement graph algorithm. It is fine when I worked in local mode, but when I worked on hadoop cluster, there always popped up some error message like: (Please see the last few lines) 2012-11-23 22:00:00,651 [main] INFO org.apache.pig.backend.hadoop.** executionengine.**mapReduceLayer.**JobControlCompiler - creating jar file Job4116346741117365374.jar 2012-11-23 22:00:09,418 [main] INFO org.apache.pig.backend.hadoop.** executionengine.**mapReduceLayer.**JobControlCompiler - jar file Job4116346741117365374.jar created 2012-11-23 22:00:09,423 [main] INFO org.apache.pig.backend.hadoop.** executionengine.**mapReduceLayer.**JobControlCompiler - Setting up multi store job 2012-11-23 22:00:09,431 [main] INFO org.apache.pig.backend.hadoop.** executionengine.**mapReduceLayer.**JobControlCompiler - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=296 2012-11-23 22:00:09,431 [main] INFO org.apache.pig.backend.hadoop.** executionengine.**mapReduceLayer.**JobControlCompiler - Neither PARALLEL nor default parallelism is set for this job. Setting number of reducers to 1 2012-11-23 22:00:09,442 [main] INFO org.apache.pig.backend.hadoop.** executionengine.**mapReduceLayer.**MapReduceLauncher - 1 map-reduce job(s) waiting for submission. 2012-11-23 22:00:09,949 [main] INFO org.apache.pig.backend.hadoop.** executionengine.**mapReduceLayer.**MapReduceLauncher - job null has failed! Stop running all dependent jobs 2012-11-23 22:00:09,949 [main] INFO org.apache.pig.backend.hadoop.** executionengine.**mapReduceLayer.**MapReduceLauncher - 100% complete 2012-11-23 22:00:09,992 [main] ERROR org.apache.pig.tools.pigstats.**SimplePigStats - ERROR 6015: During execution, encountered a Hadoop error. 2012-11-23 22:00:09,993 [main] ERROR org.apache.pig.tools.pigstats.**PigStatsUtil - 1 map reduce job(s) failed! 2012-11-23 22:00:09,994 [main] INFO org.apache.pig.tools.pigstats.**SimplePigStats - Script Statistics: HadoopVersion PigVersion UserId StartedAt FinishedAt Features 0.20.1 0.10.0 jierus 2012-11-23 21:52:38 2012-11-23 22:00:09 HASH_JOIN,GROUP_BY,DISTINCT,**FILTER,UNION Some jobs have failed! Stop running all dependent jobs Failed Jobs: JobId Alias Feature Message Outputs N/A vec_comp,vec_comp_final,vec_**comp_tmp HASH_JOIN,MULTI_QUERY Message: java.io.FileNotFoundException: File /tmp/Job4116346741117365374.**jar does not exist. at org.apache.hadoop.fs.**RawLocalFileSystem.**getFileStatus(** RawLocalFileSystem.java:361) at org.apache.hadoop.fs.**FilterFileSystem.**getFileStatus(** FilterFileSystem.java:245) at org.apache.hadoop.fs.FileUtil.**copy(FileUtil.java:192) at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(** FileSystem.java:1184) at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(** FileSystem.java:1160) at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(** FileSystem.java:1132) Does anybody know which part of my code or operation is wrong? |