Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - NullPointerException when running multiple reducers with Hadoop 0.22.0-SNAPSHOT


Copy link to this message
-
Re: NullPointerException when running multiple reducers with Hadoop 0.22.0-SNAPSHOT
madhu phatak 2011-06-30, 15:17
Can you see the logs of tasktracker for full stacktrace?

On Thu, Jun 30, 2011 at 12:24 PM, Paolo Castagna <
[EMAIL PROTECTED]> wrote:

> Hi,
> I am using Apache Whirr to setup an Hadoop cluster on EC2 using Hadoop
> 0.22.0 SNAPSHOTs (nightly) builds from Jenkins. For details, see [1,2].
> (Is there a better place where I can get nightly builds for Hadoop?)
>
> I have a Reducer which does not emit any (key,value) pairs (it generates
> only side effect files). When I run using only one reducer everything seems
> fine. If I setup multiple reducers, each one generating different side
> effect
> files, I get a NullPointerException:
>
>  WARN  Exception running child : java.lang.NullPointerException
>   at org.apache.hadoop.mapred.**TaskLogAppender.flush(**
> TaskLogAppender.java:96)
>   at org.apache.hadoop.mapred.**TaskLog.syncLogs(TaskLog.java:**239)
>   at org.apache.hadoop.mapred.**Child$4.run(Child.java:225)
>   at java.security.**AccessController.doPrivileged(**Native Method)
>   at javax.security.auth.Subject.**doAs(Subject.java:396)
>   at org.apache.hadoop.security.**UserGroupInformation.doAs(**
> UserGroupInformation.java:**1153)
>   at org.apache.hadoop.mapred.**Child.main(Child.java:217)
>
> Have you ever seen this exception/stacktrace?
>
> My driver is here [3] and the reducer is here [4].
>
> Regards,
> Paolo
>
>  [1] https://github.com/castagna/**tdbloader3/blob/master/hadoop-**
> ec2.properties<https://github.com/castagna/tdbloader3/blob/master/hadoop-ec2.properties>
>  [2] https://builds.apache.org/**view/G-L/view/Hadoop/job/**
> Hadoop-22-Build/**lastSuccessfulBuild/artifact/**
> hadoop-0.22.0-SNAPSHOT.tar.gz<https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-22-Build/lastSuccessfulBuild/artifact/hadoop-0.22.0-SNAPSHOT.tar.gz>
>  [3] https://github.com/castagna/**tdbloader3/blob/master/src/**
> main/java/com/talis/labs/tdb/**tdbloader3/ThirdDriver.java<https://github.com/castagna/tdbloader3/blob/master/src/main/java/com/talis/labs/tdb/tdbloader3/ThirdDriver.java>
>  [4] https://github.com/castagna/**tdbloader3/blob/master/src/**
> main/java/com/talis/labs/tdb/**tdbloader3/ThirdReducer.java<https://github.com/castagna/tdbloader3/blob/master/src/main/java/com/talis/labs/tdb/tdbloader3/ThirdReducer.java>
>
>