Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> NullPointerException when running multiple reducers with Hadoop 0.22.0-SNAPSHOT


Copy link to this message
-
Re: NullPointerException when running multiple reducers with Hadoop 0.22.0-SNAPSHOT
Can you see the logs of tasktracker for full stacktrace?

On Thu, Jun 30, 2011 at 12:24 PM, Paolo Castagna <
[EMAIL PROTECTED]> wrote:

> Hi,
> I am using Apache Whirr to setup an Hadoop cluster on EC2 using Hadoop
> 0.22.0 SNAPSHOTs (nightly) builds from Jenkins. For details, see [1,2].
> (Is there a better place where I can get nightly builds for Hadoop?)
>
> I have a Reducer which does not emit any (key,value) pairs (it generates
> only side effect files). When I run using only one reducer everything seems
> fine. If I setup multiple reducers, each one generating different side
> effect
> files, I get a NullPointerException:
>
>  WARN  Exception running child : java.lang.NullPointerException
>   at org.apache.hadoop.mapred.**TaskLogAppender.flush(**
> TaskLogAppender.java:96)
>   at org.apache.hadoop.mapred.**TaskLog.syncLogs(TaskLog.java:**239)
>   at org.apache.hadoop.mapred.**Child$4.run(Child.java:225)
>   at java.security.**AccessController.doPrivileged(**Native Method)
>   at javax.security.auth.Subject.**doAs(Subject.java:396)
>   at org.apache.hadoop.security.**UserGroupInformation.doAs(**
> UserGroupInformation.java:**1153)
>   at org.apache.hadoop.mapred.**Child.main(Child.java:217)
>
> Have you ever seen this exception/stacktrace?
>
> My driver is here [3] and the reducer is here [4].
>
> Regards,
> Paolo
>
>  [1] https://github.com/castagna/**tdbloader3/blob/master/hadoop-**
> ec2.properties<https://github.com/castagna/tdbloader3/blob/master/hadoop-ec2.properties>
>  [2] https://builds.apache.org/**view/G-L/view/Hadoop/job/**
> Hadoop-22-Build/**lastSuccessfulBuild/artifact/**
> hadoop-0.22.0-SNAPSHOT.tar.gz<https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-22-Build/lastSuccessfulBuild/artifact/hadoop-0.22.0-SNAPSHOT.tar.gz>
>  [3] https://github.com/castagna/**tdbloader3/blob/master/src/**
> main/java/com/talis/labs/tdb/**tdbloader3/ThirdDriver.java<https://github.com/castagna/tdbloader3/blob/master/src/main/java/com/talis/labs/tdb/tdbloader3/ThirdDriver.java>
>  [4] https://github.com/castagna/**tdbloader3/blob/master/src/**
> main/java/com/talis/labs/tdb/**tdbloader3/ThirdReducer.java<https://github.com/castagna/tdbloader3/blob/master/src/main/java/com/talis/labs/tdb/tdbloader3/ThirdReducer.java>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB