Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> failed to produce result


Copy link to this message
-
Re: failed to produce result
The logs say that the map-reduce job failed. Can you check the log files of the failed map-reduce tasks ?
You can follow the jobtracker url in the log message - http://hadoop-name02:50030/jobdetails.jsp?jobid=job_201101260357_3230

-Thejas
On 1/31/11 1:54 PM, "Dexin Wang" <[EMAIL PROTECTED]> wrote:

Hi,

I found similar problems on the web but didn't find a solution for it so I'm
asking here.

I have some pig job that has been working fine for couple of months and it
started failing. But the same job still works if run as another account. I
narrowed it a bit and found that the problematic user account can't even do
a simple DUMP.

--------------------------------------------------------------------------------------
grunt>  A = LOAD '/user/myuser1/aa' AS (f1, f2);
grunt> DESCRIBE A;
A: {f1: bytearray,f2: bytearray}
grunt> DUMP A;
2011-01-31 15:48:34,141 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
Store(hdfs://hadoop-name01/tmp/temp811847645/tmp1546738024:org.apache.pig.builtin.BinStorage)
- 1-10 Operator Key: 1-10)
2011-01-31 15:48:34,142 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 1
2011-01-31 15:48:34,142 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 1
2011-01-31 15:48:34,153 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2011-01-31 15:48:35,562 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2011-01-31 15:48:35,574 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
2011-01-31 15:48:35,578 [Thread-23] WARN  org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2011-01-31 15:48:36,077 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2011-01-31 15:48:36,176 [Thread-23] INFO
 org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
to process : 1
2011-01-31 15:48:36,176 [Thread-23] INFO
 org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1
2011-01-31 15:48:37,067 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_201101260357_3230
2011-01-31 15:48:37,067 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- More information at:
http://hadoop-name02:50030/jobdetails.jsp?jobid=job_201101260357_3230
2011-01-31 15:48:41,605 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2011-01-31 15:48:41,605 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map reduce job(s) failed!
2011-01-31 15:48:41,607 [main]* ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed to produce result in:
"hdfs://hadoop-namenode01/tmp/temp811847645/tmp1546738024"*
2011-01-31 15:48:41,607 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed!
2011-01-31 15:48:41,619 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1066: Unable to open iterator for alias A
--------------------------------------------------------------------------

While running the same LOAD, DUMP works fine with another user account. We
also confirmed there is no diskspace or quota issue on namenode. Any idea?

This is similar to this issue reported here:

http://web.archiveorange.com/archive/v/3inw3wuad4S3zjAz89y5