Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Re: Job Jar file does not exist


Copy link to this message
-
Re: Job Jar file does not exist
Do you have any cron job that cleans up /tmp directory?

On Fri, Nov 23, 2012 at 7:32 PM, Jieru Shi <[EMAIL PROTECTED]> wrote:

> HI Jagat Singh
> I have the permission to write file there.
> My scrip consists of Loops,which means several job will be created.
> It is weird such error happens irregularly, sometimes first job will fail,
> sometimes certain
> job will fail after several successful jobs.
>
>
>
> 2012/11/23 Jagat Singh <[EMAIL PROTECTED]>
>
> > First check i would do is the permission check of this temp folder.
> >
> >
> > On Sat, Nov 24, 2012 at 2:19 PM, Jieru Shi <[EMAIL PROTECTED]> wrote:
> >
> > > Hi
> > > I'm using embeded Pig to implement graph algorithm.
> > > It is fine when I worked in local mode, but when I worked on hadoop
> > > cluster,
> > > there always popped up some error message like: (Please see the last
> few
> > > lines)
> > >
> > > 2012-11-23 22:00:00,651 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler - creating jar
> file
> > > Job4116346741117365374.jar
> > > 2012-11-23 22:00:09,418 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler - jar file
> > > Job4116346741117365374.jar created
> > > 2012-11-23 22:00:09,423 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler - Setting up
> multi
> > > store job
> > > 2012-11-23 22:00:09,431 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler -
> > > BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=296
> > > 2012-11-23 22:00:09,431 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler - Neither
> PARALLEL
> > > nor default parallelism is set for this job. Setting number of reducers
> > to
> > > 1
> > > 2012-11-23 22:00:09,442 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**MapReduceLauncher - 1 map-reduce
> > job(s)
> > > waiting for submission.
> > > 2012-11-23 22:00:09,949 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**MapReduceLauncher - job null has
> > failed!
> > > Stop running all dependent jobs
> > > 2012-11-23 22:00:09,949 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**MapReduceLauncher - 100% complete
> > > 2012-11-23 22:00:09,992 [main] ERROR
> > > org.apache.pig.tools.pigstats.**SimplePigStats
> > > - ERROR 6015: During execution, encountered a Hadoop error.
> > > 2012-11-23 22:00:09,993 [main] ERROR
> > > org.apache.pig.tools.pigstats.**PigStatsUtil
> > > - 1 map reduce job(s) failed!
> > > 2012-11-23 22:00:09,994 [main] INFO
> > > org.apache.pig.tools.pigstats.**SimplePigStats
> > > - Script Statistics:
> > >
> > > HadoopVersion    PigVersion    UserId    StartedAt    FinishedAt
> Features
> > > 0.20.1    0.10.0    jierus    2012-11-23 21:52:38    2012-11-23
> 22:00:09
> > >  HASH_JOIN,GROUP_BY,DISTINCT,**FILTER,UNION
> > >
> > > Some jobs have failed! Stop running all dependent jobs
> > > Failed Jobs:
> > > JobId    Alias    Feature    Message    Outputs
> > > N/A    vec_comp,vec_comp_final,vec_**comp_tmp HASH_JOIN,MULTI_QUERY
> > >  Message: java.io.FileNotFoundException: File
> > > /tmp/Job4116346741117365374.**jar
> > > does not exist.
> > >     at org.apache.hadoop.fs.**RawLocalFileSystem.**getFileStatus(**
> > > RawLocalFileSystem.java:361)
> > >     at org.apache.hadoop.fs.**FilterFileSystem.**getFileStatus(**
> > > FilterFileSystem.java:245)
> > >     at org.apache.hadoop.fs.FileUtil.**copy(FileUtil.java:192)
> > >     at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(**
> > > FileSystem.java:1184)
> > >     at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(**
> > > FileSystem.java:1160)
> > >     at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(**
> > > FileSystem.java:1132)
> > >
> > > Does anybody know which part of my code or operation is wrong?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB