Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Intermittent NullPointerException


Copy link to this message
-
Intermittent NullPointerException
Malcolm Tye 2012-11-12, 15:14
Hi,

    I'm running Pig 0.10.0 in local mode on some small text files. There is
no intention to run it on Hadoop at all. We have a job that runs every 5
minutes and about 3% of the time, the job fails with the error below. It
happens at random places within the Pig Script.

 

2012-10-19 14:15:37,719 [Thread-15] WARN
org.apache.hadoop.mapred.LocalJobRunner - job_local_0004
java.lang.NullPointerException
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.processInput(PhysicalOperator.java:286)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperat
ors.POProject.getNext(POProject.java:158)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperat
ors.POProject.getNext(POProject.java:360)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.getNext(PhysicalOperator.java:330)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POForEach.processPlan(POForEach.java:332)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POForEach.getNext(POForEach.java:284)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.processInput(PhysicalOperator.java:290)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POFilter.getNext(POFilter.java:95)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.processInput(PhysicalOperator.java:290)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POForEach.getNext(POForEach.java:233)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.processInput(PhysicalOperator.java:290)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POLocalRearrange.getNext(POLocalRearrange.java:256)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POUnion.getNext(POUnion.java:165)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBa
se.runPipeline(PigGenericMapBase.java:271)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBa
se.map(PigGenericMapBase.java:266)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBa
se.map(PigGenericMapBase.java:64)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
        at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

 

In the Pig Log, I get

 

ERROR 2244: Job failed, hadoop does not return any error message

org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job
failed, hadoop does not return any error message
        at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193
)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165
)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:555)
        at org.apache.pig.Main.main(Main.java:111)
==============================================================================
 

Pig script is attached.

 

Any help gratefully received

 

Thanks

 

Malc