Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> failed to produce result


Copy link to this message
-
failed to produce result
Hi,

I found similar problems on the web but didn't find a solution for it so I'm
asking here.

I have some pig job that has been working fine for couple of months and it
started failing. But the same job still works if run as another account. I
narrowed it a bit and found that the problematic user account can't even do
a simple DUMP.

--------------------------------------------------------------------------------------
grunt>  A = LOAD '/user/myuser1/aa' AS (f1, f2);
grunt> DESCRIBE A;
A: {f1: bytearray,f2: bytearray}
grunt> DUMP A;
2011-01-31 15:48:34,141 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
Store(hdfs://hadoop-name01/tmp/temp811847645/tmp1546738024:org.apache.pig.builtin.BinStorage)
- 1-10 Operator Key: 1-10)
2011-01-31 15:48:34,142 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 1
2011-01-31 15:48:34,142 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 1
2011-01-31 15:48:34,153 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2011-01-31 15:48:35,562 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2011-01-31 15:48:35,574 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
2011-01-31 15:48:35,578 [Thread-23] WARN  org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2011-01-31 15:48:36,077 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2011-01-31 15:48:36,176 [Thread-23] INFO
 org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
to process : 1
2011-01-31 15:48:36,176 [Thread-23] INFO
 org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1
2011-01-31 15:48:37,067 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_201101260357_3230
2011-01-31 15:48:37,067 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- More information at:
http://hadoop-name02:50030/jobdetails.jsp?jobid=job_201101260357_3230
2011-01-31 15:48:41,605 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2011-01-31 15:48:41,605 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map reduce job(s) failed!
2011-01-31 15:48:41,607 [main]* ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed to produce result in:
"hdfs://hadoop-namenode01/tmp/temp811847645/tmp1546738024"*
2011-01-31 15:48:41,607 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed!
2011-01-31 15:48:41,619 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1066: Unable to open iterator for alias A
--------------------------------------------------------------------------

While running the same LOAD, DUMP works fine with another user account. We
also confirmed there is no diskspace or quota issue on namenode. Any idea?

This is similar to this issue reported here:

http://web.archiveorange.com/archive/v/3inw3wuad4S3zjAz89y5
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB