Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Commands not working properly when stored in pig file

Copy link to this message
Commands not working properly when stored in pig file
Mix Nin 2013-03-27, 21:58
Sorry for posting same issue multiple times

I  wrote a pig script as follows and stored it in x.pig file

Data = LOAD '/....' as (,,,, )
NoNullData= FILTER Data by qe is not null;
STORE (foreach (group NoNullData all) generate flatten($1))  into
evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)

I executed the command as follows

pig  -f x.pig -param inputDatePig=03272013
And  finally it says exp/03272013 tough the directory exists as it gets
created in STORE command.

What is wrong in this
This is the error I get

- 32% complete
2013-03-27 14:38:35,568 [main] INFO
- 50% complete
2013-03-27 14:38:45,731 [main] INFO
- job null has failed! Stop running all dependent jobs
2013-03-27 14:38:45,731 [main] INFO
- 100% complete
2013-03-27 14:38:45,734 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input
path does not exist: hdfs://user/lnindrakrishna/exp/03272013/part-r-00000

But when I remove second LOAD command, everything runs fine . Why does it
throw job "null has failed! Stop running all dependent jobs"
Johnny Zhang 2013-03-27, 22:16
Mix Nin 2013-03-27, 22:45
Mix Nin 2013-03-27, 23:07
Johnny Zhang 2013-03-28, 03:15
Johnny Zhang 2013-03-28, 16:19