Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - join operation fails on big data set


Copy link to this message
-
Re: join operation fails on big data set
Cheolsoo Park 2013-04-12, 18:25
Hi Mua,

I guess you misunderstood me. The pig_****.log file is not a task log.

You should look for task logs on data nodes where your task tracker ran.
Here is some explanation regarding various log files in Hadoop and where to
find them:

http://blog.cloudera.com/blog/2009/09/apache-hadoop-log-files-where-to-find-them-in-cdh-and-what-info-they-contain/

Thanks,
Cheolsoo
On Fri, Apr 12, 2013 at 10:27 AM, Mua Ban <[EMAIL PROTECTED]> wrote:

> Thank you very much for your reply.
>
> Below is the stack log in the pig_****.log file
>
> Can you please give me some suggestions?
>
> -Mua
> ------------------
> Backend error message
> ---------------------
> Task attempt_201304081613_0048_r_000001_0 failed to report status for 601
> seconds. Killing!
>
> Pig Stack Trace
> ---------------
> ERROR 2997: Unable to recreate exception from backed error: Task
> attempt_201304081613_0048_r_000001_0 failed to report status for 601
> seconds. Killing!
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 2997: Unable to
> recreate exception from backed error: Task
> attempt_201304081613_0048_r_000001_0 failed to report status for 601
> seconds. Killing!
>         at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:217)
>         at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:152)
>         at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:383)
>         at org.apache.pig.PigServer.launchPlan(PigServer.java:1270)
>         at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1255)
>         at org.apache.pig.PigServer.execute(PigServer.java:1245)
>         at org.apache.pig.PigServer.executeBatch(PigServer.java:362)
>         at
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
>         at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)
>         at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>         at org.apache.pig.Main.run(Main.java:555)
>         at org.apache.pig.Main.main(Main.java:111)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
>         at java.lang.reflect.Method.invoke(Method.java:611)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> ===============================================================================>
>
>
>
>
> On Fri, Apr 12, 2013 at 11:29 AM, Cheolsoo Park <[EMAIL PROTECTED]
> >wrote:
>
> > Did you look at task logs to see why those tasks failed? Since it's a
> > back-end error, the console output doesn't tell you much. Task logs
> should
> > have a stack trace that shows why it failed, and you can go from there.
> >
> >
> >
> > On Fri, Apr 12, 2013 at 8:18 AM, Mua Ban <[EMAIL PROTECTED]> wrote:
> >
> > > Hi,
> > >
> > > I am very new to PIG/Hadoop, I just started writing my first PIG
> script a
> > > couple days ago. I ran into this problem.
> > >
> > > My cluster has 9 nodes. I have to join two data sets big and small,
> each
> > is
> > > collected for 4 weeks. I first take two subsets of my data set (which
> is
> > > for the first week of data), let's call them B1 and S1 for big and
> small
> > > data sets of the first week. The entire data sets of 4 weeks is B4 and
> > S4.
> > >
> > > I ran my script on my cluster to join B1 and S1 and everything is
> fine. I
> > > got my joined data. However, when I ran my script to join B4 and S4,
> the
> > > script failed. B4 is 39GB, S4 is 300MB. B4 is skewed, some id appears
> > more
> > > frequently than others. I tried both 'using skewed' and 'using
> > replicated'