Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Intermittent NullPointerException


Copy link to this message
-
Re: Intermittent NullPointerException
Hi Malcolm,

I have been running your script with 10M rows for a half day but couldn't
reproduce your error. So my analysis may be baseless here.

That being said, it looks like a race condition to me. The callstack in the
log shows below:

Caused by: java.lang.NullPointerException
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:286)

Now if you look at PhysicalOperator.java:286, it's like this:

if(reporter!=null) {
    reporter.progress(); ---> NullPointerException is thrown here
}

So 'reporter' became null between 'if(reporter!=null)' and
'reporter.progress()'.

Given that 'reporter' is a static field, this is totally possible.

public static PigProgressable reporter;

Even though you're setting default_parallel to 1, it only controls the
number of reducers, and the number of mappers is determined by the size of
input data. So you will still run multiple mapper threads in parallel in
LocalJobRunner, and they might be stepping into each other.

One possible fix is probably changing reporter to a thread local variable.
I will send a patch that does this to your email address. I based it to
branch-0.10, so you should be able to apply it cleanly to the 0.10 source
tarball running:

patch -p0 -i <patch file>

Can you please try to apply the patch, rebuild pig and see if that fixes
your problem? If this does, I will try to write a unit test case and commit
the fix upstream as well.

Thanks,
Cheolsoo

On Wed, Nov 14, 2012 at 4:32 AM, Malcolm Tye <[EMAIL PROTECTED]>wrote:

> Hi,
>         Looks like zip files get rejected. Here's the log file unzipped
>
>
> Malc
>
>
> -----Original Message-----
> From: Malcolm Tye [mailto:[EMAIL PROTECTED]]
> Sent: 14 November 2012 12:01
> To: '[EMAIL PROTECTED]'
> Subject: RE: Intermittent NullPointerException
>
> Hi Cheolsoo,
>                 Even with the recompiled Pig, we still see the error. He's
> a
> debug log from Pig. It doesn't seem to give any more information.
>
> Any ideas ?
>
>
> Thanks
>
> Malc
>
>
> -----Original Message-----
> From: Malcolm Tye [mailto:[EMAIL PROTECTED]]
> Sent: 13 November 2012 12:58
> To: '[EMAIL PROTECTED]'
> Subject: RE: Intermittent NullPointerException
>
> Hi Cheolsoo,
>                 I tried setting default_parallel to 1 to rule out parallel
> processing, but the problem still happened.
>
> I've recompiled Pig and have put that into the test environment with the
> debug option set.
>
> I don't have recreate steps that fail every time. When the problem occurs,
> we can run the same script again on the input file  and the file gets
> processed OK the next time !
>
> Thanks
>
> Malc
>
>
> -----Original Message-----
> From: Cheolsoo Park [mailto:[EMAIL PROTECTED]]
> Sent: 12 November 2012 23:00
> To: [EMAIL PROTECTED]
> Subject: Re: Intermittent NullPointerException
>
> Hi Malcolm,
>
> If you're not running in parallel, it may be a different issue. But I am
> surprised that Pig 0.10 local mode fails Intermittently like you describe
> w/o parallelism. You might have discovered a real issue. If you could
> provide steps that reproduce the error, that would be great!
>
> >> How do I tell which pig jar file I'm using currently ?
>
> "pig -secretDebugCmd" will show which pig jar file in file system is picked
> up. For example, it shows the following output for me:
>
> /usr/bin/hadoop jar /home/cheolsoo/pig-svn/bin/../pig-withouthadoop.jar
>
> Thanks,
> Cheolsoo
>
> On Mon, Nov 12, 2012 at 2:46 PM, Malcolm Tye
> <[EMAIL PROTECTED]>wrote:
>
> > Hi Cheolsoo,
> >                 I'm not specifically setting default_parallel in my
> > script anywhere and I see this in the log file :-
> >
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> > lCompi ler - Neither PARALLEL nor default parallelism is set for this
> > job. Setting number of reducers to 1
> >
> > So I guess I'm not using parallel. Is it worth trying to compile Pig
> > to use the Hadoop 0.23.x LocalJobRunner ? How do I tell which pig jar