|
|
-
Issue with Reduce Side join using datajoin packageVikas Jadhav 2013-01-29, 12:00
I am using Hadoop 1.0.3
I am getting following Error 13/01/29 06:55:19 INFO mapred.JobClient: Task Id : attempt_201301290120_0006_r_000000_0, Status : FAILED java.lang.NullPointerException at MyJoin$TaggedWritable.readFields(MyJoin.java:101) at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67) at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40) at org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:1271) at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:1211) at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:249) at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:245) at org.apache.hadoop.contrib.utils.join.DataJoinReducerBase.regroup(DataJoinReducerBase.java:106) at org.apache.hadoop.contrib.utils.join.DataJoinReducerBase.reduce(DataJoinReducerBase.java:129) at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:519) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) It is poiting to String dataClz = in.readUTF(); this line in readFields * public * *void* readFields( DataInput in) *throws* IOException { *this*.tag.readFields( in); //String dataClz = in.readUTF(); String dataClz = in.readUTF(); ----> error log show this line is culprit *try* * * { //try - catch is needed because the " error: unreported exception //ClassNotFoundException; must be caught or declared to be thrown" //is "raised" from compiler *if*( *this*.data == *null* || !*this*.data.getClass().getName().equals( dataClz)) { //this line of code "raises" the compile error mentioned above *this*.data = (Writable) ReflectionUtils.*newInstance*( Class.*forName*( dataClz), *null*); } *this*.data.readFields( in); } *catch*( ClassNotFoundException cnfe) { System.*out*.println( "Problem in TaggedWritable class, method readFields." ); } }//end readFields -- * * * Thanx and Regards* * Vikas Jadhav* |