Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Issue with Reduce Side join using datajoin package


Copy link to this message
-
Issue with Reduce Side join using datajoin package
Vikas Jadhav 2013-01-29, 12:00
I am using Hadoop 1.0.3

I am getting following Error
13/01/29 06:55:19 INFO mapred.JobClient: Task Id :
attempt_201301290120_0006_r_000000_0, Status : FAILED
java.lang.NullPointerException
        at MyJoin$TaggedWritable.readFields(MyJoin.java:101)
        at
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
        at
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
        at
org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:1271)
        at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:1211)
        at
org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:249)
        at
org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:245)
        at
org.apache.hadoop.contrib.utils.join.DataJoinReducerBase.regroup(DataJoinReducerBase.java:106)
        at
org.apache.hadoop.contrib.utils.join.DataJoinReducerBase.reduce(DataJoinReducerBase.java:129)
        at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:519)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
It is poiting to  String dataClz = in.readUTF(); this line in readFields
*

public
*

*void* readFields( DataInput in) *throws* IOException

{

*this*.tag.readFields( in);

//String dataClz = in.readUTF();

String dataClz = in.readUTF(); ----> error log show this line is culprit

*try*
* *

{

//try - catch is needed because the " error: unreported exception

//ClassNotFoundException; must be caught or declared to be thrown"

//is "raised" from compiler

*if*( *this*.data == *null* || !*this*.data.getClass().getName().equals(
dataClz))

{

//this line of code "raises" the compile error mentioned above

*this*.data = (Writable) ReflectionUtils.*newInstance*( Class.*forName*(
dataClz), *null*);

}

*this*.data.readFields( in);

}

*catch*( ClassNotFoundException cnfe)

{

System.*out*.println( "Problem in TaggedWritable class, method readFields."
);

}

}//end readFields
--
*
*
*

Thanx and Regards*
* Vikas Jadhav*