Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> pig12 job stuck in infinite loop


Copy link to this message
-
pig12 job stuck in infinite loop
Hi
I launched the attached pig job on pig-12 with hadoop MRv1 with the
attached data, but the FILTER function causes the job to get stuck in an
infinite loop.

pig -p pPeriod=201312 -f test.pig

The thread in question seems to be stuck forever inside while loop of
runPipeline method.

stack trace:

"main" prio=10 tid=0x00007fd74800b000 nid=0x2f63 runnable
[0x00007fd750d50000]
   java.lang.Thread.State: RUNNABLE
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.
relationalOperators.POForEach.getNextTuple(POForEach.java:217)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.map(PigGenericMapBase.java:277)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.map(PigGenericMapBase.java:64)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:680)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:346)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:282)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at
org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1117)
    at org.apache.hadoop.mapred.Child.main(Child.java:271)
org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/
PigGenericMapBase.java:

protected void *runPipeline*(PhysicalOperator leaf) throws IOException,
InterruptedException {
        while(true){
            Result res = leaf.getNext(DUMMYTUPLE);
            if(res.returnStatus==POStatus.STATUS_OK){
                collect(outputCollector,(Tuple)res.result);
                continue;
            }
....

Whats the suggested code fix here?
Thanks,
Suhas.

 20100101|1 WEEK ENDING 01/02/10|2010-Jan-02|201001|1468
20100102|1 WEEK ENDING 01/09/10|2010-Jan-09|201001|1469
20100103|1 WEEK ENDING 01/16/10|2010-Jan-16|201001|1470
20100104|1 WEEK ENDING 01/23/10|2010-Jan-23|201001|1471
20100205|1 WEEK ENDING 01/30/10|2010-Jan-30|201002|1472
20100206|1 WEEK ENDING 02/06/10|2010-Feb-06|201002|1473
20100207|1 WEEK ENDING 02/13/10|2010-Feb-13|201002|1474
20100208|1 WEEK ENDING 02/20/10|2010-Feb-20|201002|1475
20100309|1 WEEK ENDING 02/27/10|2010-Feb-27|201003|1476
20100310|1 WEEK ENDING 03/06/10|2010-Mar-06|201003|1477
20100311|1 WEEK ENDING 03/13/10|2010-Mar-13|201003|1478
20100312|1 WEEK ENDING 03/20/10|2010-Mar-20|201003|1479
20100413|1 WEEK ENDING 03/27/10|2010-Mar-27|201004|1480
20100414|1 WEEK ENDING 04/03/10|2010-Apr-03|201004|1481
20100415|1 WEEK ENDING 04/10/10|2010-Apr-10|201004|1482
20100416|1 WEEK ENDING 04/17/10|2010-Apr-17|201004|1483
20100517|1 WEEK ENDING 04/24/10|2010-Apr-24|201005|1484
20100518|1 WEEK ENDING 05/01/10|2010-May-01|201005|1485
20100519|1 WEEK ENDING 05/08/10|2010-May-08|201005|1486
20100520|1 WEEK ENDING 05/15/10|2010-May-15|201005|1487
20100621|1 WEEK ENDING 05/22/10|2010-May-22|201006|1488
20100622|1 WEEK ENDING 05/29/10|2010-May-29|201006|1489
20100623|1 WEEK ENDING 06/05/10|2010-Jun-05|201006|1490
20100624|1 WEEK ENDING 06/12/10|2010-Jun-12|201006|1491
20100725|1 WEEK ENDING 06/19/10|2010-Jun-19|201007|1492
20100726|1 WEEK ENDING 06/26/10|2010-Jun-26|201007|1493
20100727|1 WEEK ENDING 07/03/10|2010-Jul-03|201007|1494
20100728|1 WEEK ENDING 07/10/10|2010-Jul-10|201007|1495
20100829|1 WEEK ENDING 07/17/10|2010-Jul-17|201008|1496
20100830|1 WEEK ENDING 07/24/10|2010-Jul-24|201008|1497
20100831|1 WEEK ENDING 07/31/10|2010-Jul-31|201008|1498
20100832|1 WEEK ENDING 08/07/10|2010-Aug-07|201008|1499
20100933|1 WEEK ENDING 08/14/10|2010-Aug-14|201009|1500
20100934|1 WEEK ENDING 08/21/10|2010-Aug-21|201009|1501
20100935|1 WEEK ENDING 08/28/10|2010-Aug-28|201009|1502
20100936|1 WEEK ENDING 09/04/10|2010-Sep-04|201009|1503
20101037|1 WEEK ENDING 09/11/10|2010-Sep-11|201010|1504
20101038|1 WEEK ENDING 09/18/10|2010-Sep-18|201010|1505
20101039|1 WEEK ENDING 09/25/10|2010-Sep-25|201010|1506
20101040|1 WEEK ENDING 10/02/10|2010-Oct-02|201010|1507
20101141|1 WEEK ENDING 10/09/10|2010-Oct-09|201011|1508
20101142|1 WEEK ENDING 10/16/10|2010-Oct-16|201011|1509
20101143|1 WEEK ENDING 10/23/10|2010-Oct-23|201011|1510
20101144|1 WEEK ENDING 10/30/10|2010-Oct-30|201011|1511
20101245|1 WEEK ENDING 11/06/10|2010-Nov-06|201012|1512
20101246|1 WEEK ENDING 11/13/10|2010-Nov-13|201012|1513
20101247|1 WEEK ENDING 11/20/10|2010-Nov-20|201012|1514
20101248|1 WEEK ENDING 11/27/10|2010-Nov-27|201012|1515
20101349|1 WEEK ENDING 12/04/10|2010-Dec-04|201013|1516
20101350|1 WEEK ENDING 12/11/10|2010-Dec-11|201013|1517
20101351|1 WEEK ENDING 12/18/10|2010-Dec-18|201013|1518
20101352|1 WEEK ENDING 12/25/10|2010-Dec-25|201013|1519
20110101|1 WEEK ENDING 01/01/11|2011-Jan-01|201101|1520
20110102|1 WEEK ENDING 01/08/11|2011-Jan-08|201101|1521
20110103|1 WEEK ENDING 01/15/11|2011-Jan-15|201101|1522
20110104|1 WEEK ENDING 01/22/11|2011-Jan-22|201101|1523
20110205|1 WEEK ENDING 01/29/11|2011-Jan-29|201102|1524
20110206|1 WEEK ENDING 02/05/11|2011-Feb-05|201102|1525
20110207|1 WEEK ENDING 02/12/11|2011-Feb-12|201102|1526
20110208|1 WEEK ENDING 02/19/11|2011-Feb-19|201102|1527
20110309|1 WEEK ENDING 02/26/11|2011-Feb-26|201103|1528
20110310|1 WEEK ENDING 03/05/11|2011-Mar-05|201103|1529
20110311|1 WEEK ENDING 03/12/11|2011-Mar-12|201103|1530
20110312|1 WEEK ENDING 03/19/11|2011-Mar-19|201103|1531
20110413|1 WEEK ENDING 03/26/11|2011-Mar-26|201104|1532
20110414|1 WEEK ENDING 04/02/11|2011-Apr-02|201104|1533
20110415|1 WEEK ENDING 04/09/11|2011-Apr-09|201104|1534
20110416|1 WEEK ENDING 04/16/11|2011-Apr-16|201104|1535
20110517|1 WEEK ENDING 04/23/11|2011-Apr-23|201105|1536
20110518|1 WEEK ENDING 04/30/11|2011-Apr-30|201105|1537
20110519|1 WEEK ENDING 05/07/11|2011-May-07|201105|1538
20110520|1 WEEK ENDING 05/14/11|2011-May-14|201105|1539
20110621|1 WEEK ENDING 05/21/11|2011-May-21|201106|1