Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Commands not working properly when stored in pig file


Copy link to this message
-
Commands not working properly when stored in pig file
Sorry for posting same issue multiple times

I  wrote a pig script as follows and stored it in x.pig file

Data = LOAD '/....' as (,,,, )
NoNullData= FILTER Data by qe is not null;
STORE (foreach (group NoNullData all) generate flatten($1))  into
'exp/$inputDatePig';
evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)

I executed the command as follows

pig  -f x.pig -param inputDatePig=03272013
And  finally it says exp/03272013 tough the directory exists as it gets
created in STORE command.

What is wrong in this
This is the error I get

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 32% complete
2013-03-27 14:38:35,568 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 50% complete
2013-03-27 14:38:45,731 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job null has failed! Stop running all dependent jobs
2013-03-27 14:38:45,731 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2013-03-27 14:38:45,734 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input
path does not exist: hdfs://user/lnindrakrishna/exp/03272013/part-r-00000

But when I remove second LOAD command, everything runs fine . Why does it
throw job "null has failed! Stop running all dependent jobs"
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB