Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Intermittent NullPointerException


Copy link to this message
-
Re: Intermittent NullPointerException
Hi Malcolm,

How do you run your script? Do you run your script in parallel? Hadoop
1.0.x LocalJobRunner is not thread-safe, and Pig is by default built with
Hadoop 1.0.x. I have seen a similar problem before (
https://issues.apache.org/jira/browse/PIG-2852).

If you're running your script in parallel, one workaround is to use Hdoop
0.23.x LocalJobRunner, which is thread-safe. You can do the following:
- If you're using the standalone pig.jar, please download the Pig source
tarball and run "ant clean jar -Dhadoopversion=23" to build pig.jar.
- If you're using installed Hadoop with pig-withouthadoop.jar, please
install Hadoop 0.23.x, download the Pig source tarball, and run "ant clean
jar-withouthadoop -Dhadoopversion=23" to build pig-withouthadoop.jar.

Hope this is helpful.

Thanks,
Cheolsoo

On Mon, Nov 12, 2012 at 7:14 AM, Malcolm Tye <[EMAIL PROTECTED]>wrote:

> Hi,****
>
>     I’m running Pig 0.10.0 in local mode on some small text files. There
> is no intention to run it on Hadoop at all. We have a job that runs every 5
> minutes and about 3% of the time, the job fails with the error below. It
> happens at random places within the Pig Script.****
>
> ** **
>
> 2012-10-19 14:15:37,719 [Thread-15] WARN
> org.apache.hadoop.mapred.LocalJobRunner - job_local_0004
> java.lang.NullPointerException
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:286)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:158)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:360)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:330)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:95)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:233)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POUnion.getNext(POUnion.java:165)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:271)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:266)
>
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
>
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>         at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)**
> **
>
> ** **
>
> In the Pig Log, I get****
>
> ** **
>
> ERROR 2244: Job failed, hadoop does not return any error message
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job
> failed, hadoop does not return any error message