Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> pig12 job stuck in infinite loop


Copy link to this message
-
pig12 job stuck in infinite loop
Hi
I launched the attached pig job on pig-12 with hadoop MRv1 with the
attached data, but the FILTER function causes the job to get stuck in an
infinite loop.

pig -p pPeriod=201312 -f test.pig

The thread in question seems to be stuck forever inside while loop of
runPipeline method.

stack trace:

"main" prio=10 tid=0x00007fd74800b000 nid=0x2f63 runnable
[0x00007fd750d50000]
   java.lang.Thread.State: RUNNABLE
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.
relationalOperators.POForEach.getNextTuple(POForEach.java:217)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.map(PigGenericMapBase.java:277)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.map(PigGenericMapBase.java:64)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:680)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:346)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:282)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at
org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1117)
    at org.apache.hadoop.mapred.Child.main(Child.java:271)
org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/
PigGenericMapBase.java:

protected void *runPipeline*(PhysicalOperator leaf) throws IOException,
InterruptedException {
        while(true){
            Result res = leaf.getNext(DUMMYTUPLE);
            if(res.returnStatus==POStatus.STATUS_OK){
                collect(outputCollector,(Tuple)res.result);
                continue;
            }
....

Whats the suggested code fix here?
Thanks,
Suhas.

 20100101|1 WEEK ENDING 01/02/10|2010-Jan-02|201001|1468
20100102|1 WEEK ENDING 01/09/10|2010-Jan-09|201001|1469
20100103|1 WEEK ENDING 01/16/10|2010-Jan-16|201001|1470
20100104|1 WEEK ENDING 01/23/10|2010-Jan-23|201001|1471
20100205|1 WEEK ENDING 01/30/10|2010-Jan-30|201002|1472
20100206|1 WEEK ENDING 02/06/10|2010-Feb-06|201002|1473
20100207|1 WEEK ENDING 02/13/10|2010-Feb-13|201002|1474
20100208|1 WEEK ENDING 02/20/10|2010-Feb-20|201002|1475
20100309|1 WEEK ENDING 02/27/10|2010-Feb-27|201003|1476
20100310|1 WEEK ENDING 03/06/10|2010-Mar-06|201003|1477
20100311|1 WEEK ENDING 03/13/10|2010-Mar-13|201003|1478
20100312|1 WEEK ENDING 03/20/10|2010-Mar-20|201003|1479
20100413|1 WEEK ENDING 03/27/10|2010-Mar-27|201004|1480
20100414|1 WEEK ENDING 04/03/10|2010-Apr-03|201004|1481
20100415|1 WEEK ENDING 04/10/10|2010-Apr-10|201004|1482
20100416|1 WEEK ENDING 04/17/10|2010-Apr-17|201004|1483
20100517|1 WEEK ENDING 04/24/10|2010-Apr-24|201005|1484
20100518|1 WEEK ENDING 05/01/10|2010-May-01|201005|1485
20100519|1 WEEK ENDING 05/08/10|2010-May-08|201005|1486
20100520|1 WEEK ENDING 05/15/10|2010-May-15|201005|1487
20100621|1 WEEK ENDING 05/22/10|2010-May-22|201006|1488
20100622|1 WEEK ENDING 05/29/10|2010-May-29|201006|1489
20100623|1 WEEK ENDING 06/05/10|2010-Jun-05|201006|1490
20100624|1 WEEK ENDING 06/12/10|2010-Jun-12|201006|1491
20100725|1 WEEK ENDING 06/19/10|2010-Jun-19|201007|1492
20100726|1 WEEK ENDING 06/26/10|2010-Jun-26|201007|1493
20100727|1 WEEK ENDING 07/03/10|2010-Jul-03|201007|1494
20100728|1 WEEK ENDING 07/10/10|2010-Jul-10|201007|1495
20100829|1 WEEK ENDING 07/17/10|2010-Jul-17|201008|1496
20100830|1 WEEK ENDING 07/24/10|2010-Jul-24|201008|1497
20100831|1 WEEK ENDING 07/31/10|2010-Jul-31|201008|1498
20100832|1 WEEK ENDING 08/07/10|2010-Aug-07|201008|1499
20100933|1 WEEK ENDING 08/14/10|2010-Aug-14|201009|1500
20100934|1 WEEK ENDING 08/21/10|2010-Aug-21|201009|1501
20100935|1 WEEK ENDING 08/28/10|2010-Aug-28|201009|1502
20100936|1 WEEK ENDING 09/04/10|2010-Sep-04|201009|1503
20101037|1 WEEK ENDING 09/11/10|2010-Sep-11|201010|1504
20101038|1 WEEK ENDING 09/18/10|2010-Sep-18|201010|1505
20101039|1 WEEK ENDING 09/25/10|2010-Sep-25|201010|1506
20101040|1 WEEK ENDING 10/02/10|2010-Oct-02|201010|1507
20101141|1 WEEK ENDING 10/09/10|2010-Oct-09|201011|1508
20101142|1 WEEK ENDING 10/16/10|2010-Oct-16|201011|1509
20101143|1 WEEK ENDING 10/23/10|2010-Oct-23|201011|1510
20101144|1 WEEK ENDING 10/30/10|2010-Oct-30|201011|1511
20101245|1 WEEK ENDING 11/06/10|2010-Nov-06|201012|1512
20101246|1 WEEK ENDING 11/13/10|2010-Nov-13|201012|1513
20101247|1 WEEK ENDING 11/20/10|2010-Nov-20|201012|1514
20101248|1 WEEK ENDING 11/27/10|2010-Nov-27|201012|1515
20101349|1 WEEK ENDING 12/04/10|2010-Dec-04|201013|1516
20101350|1 WEEK ENDING 12/11/10|2010-Dec-11|201013|1517
20101351|1 WEEK ENDING 12/18/10|2010-Dec-18|201013|1518
20101352|1 WEEK ENDING 12/25/10|2010-Dec-25|201013|1519
20110101|1 WEEK ENDING 01/01/11|2011-Jan-01|201101|1520
20110102|1 WEEK ENDING 01/08/11|2011-Jan-08|201101|1521
20110103|1 WEEK ENDING 01/15/11|2011-Jan-15|201101|1522
20110104|1 WEEK ENDING 01/22/11|2011-Jan-22|201101|1523
20110205|1 WEEK ENDING 01/29/11|2011-Jan-29|201102|1524
20110206|1 WEEK ENDING 02/05/11|2011-Feb-05|201102|1525
20110207|1 WEEK ENDING 02/12/11|2011-Feb-12|201102|1526
20110208|1 WEEK ENDING 02/19/11|2011-Feb-19|201102|1527
20110309|1 WEEK ENDING 02/26/11|2011-Feb-26|201103|1528
20110310|1 WEEK ENDING 03/05/11|2011-Mar-05|201103|1529
20110311|1 WEEK ENDING 03/12/11|2011-Mar-12|201103|1530
20110312|1 WEEK ENDING 03/19/11|2011-Mar-19|201103|1531
20110413|1 WEEK ENDING 03/26/11|2011-Mar-26|201104|1532
20110414|1 WEEK ENDING 04/02/11|2011-Apr-02|201104|1533
20110415|1 WEEK ENDING 04/09/11|2011-Apr-09|201104|1534
20110416|1 WEEK ENDING 04/16/11|2011-Apr-16|201104|1535
20110517|1 WEEK ENDING 04/23/11|2011-Apr-23|201105|1536
20110518|1 WEEK ENDING 04/30/11|2011-Apr-30|201105|1537
20110519|1 WEEK ENDING 05/07/11|2011-May-07|201105|1538
20110520|1 WEEK ENDING 05/14/11|2011-May-14|201105|1539
20110621|1 WEEK ENDING 05/21/11|2011-May-21|201106|1
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB