Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Commands not working properly when stored in pig file


+
Mix Nin 2013-03-27, 21:58
+
Johnny Zhang 2013-03-27, 22:16
Copy link to this message
-
Re: Commands not working properly when stored in pig file
Mix Nin 2013-03-27, 22:45
yes the file exists in HDFS.
On Wed, Mar 27, 2013 at 3:16 PM, Johnny Zhang <[EMAIL PROTECTED]> wrote:

> Mix,
> 'null' is the failed job ID. From what I can tell, there is only one STORE
> command and it actually fail, so MapReduceLauncher tries to stop
> all dependent jobs, that's why the message is trhown. Can you double check
> if the file exists in HDFS?
>
> Johnny
>
>
> On Wed, Mar 27, 2013 at 2:58 PM, Mix Nin <[EMAIL PROTECTED]> wrote:
>
> > Sorry for posting same issue multiple times
> >
> > I  wrote a pig script as follows and stored it in x.pig file
> >
> > Data = LOAD '/....' as (,,,, )
> > NoNullData= FILTER Data by qe is not null;
> > STORE (foreach (group NoNullData all) generate flatten($1))  into
> > 'exp/$inputDatePig';
> >
> >
> > evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
> >
> >
> >
> > I executed the command as follows
> >
> > pig  -f x.pig -param inputDatePig=03272013
> >
> >
> > And  finally it says exp/03272013 tough the directory exists as it gets
> > created in STORE command.
> >
> > What is wrong in this
> >
> >
> > This is the error I get
> >
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 32% complete
> > 2013-03-27 14:38:35,568 [main] INFO
> >
> >
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 50% complete
> > 2013-03-27 14:38:45,731 [main] INFO
> >
> >
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - job null has failed! Stop running all dependent jobs
> > 2013-03-27 14:38:45,731 [main] INFO
> >
> >
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 100% complete
> > 2013-03-27 14:38:45,734 [main] ERROR
> > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
> > recreate exception from backend error:
> > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input
> > path does not exist: hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
> >
> >
> >
> > But when I remove second LOAD command, everything runs fine . Why does it
> > throw job "null has failed! Stop running all dependent jobs"
> >
>
+
Mix Nin 2013-03-27, 23:07
+
Johnny Zhang 2013-03-28, 03:15
+
MARCOS MEDRADO RUBINELLI 2013-03-28, 10:08
+
Johnny Zhang 2013-03-28, 16:19