Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Re: Issue with Reduce Side join using datajoin package


Copy link to this message
-
Re: Issue with Reduce Side join using datajoin package
Vinod Kumar Vavilapalli 2013-01-29, 20:15
Seems like a bug in your code, can you share the source here?

+Vinod
On Tue, Jan 29, 2013 at 4:00 AM, Vikas Jadhav <[EMAIL PROTECTED]>wrote:

> I am using Hadoop 1.0.3
>
> I am getting following Error
>
>
> 13/01/29 06:55:19 INFO mapred.JobClient: Task Id :
> attempt_201301290120_0006_r_000000_0, Status : FAILED
> java.lang.NullPointerException
>         at MyJoin$TaggedWritable.readFields(MyJoin.java:101)
>         at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
>         at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
>         at
> org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:1271)
>         at
> org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:1211)
>         at
> org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:249)
>         at
> org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:245)
>         at
> org.apache.hadoop.contrib.utils.join.DataJoinReducerBase.regroup(DataJoinReducerBase.java:106)
>         at
> org.apache.hadoop.contrib.utils.join.DataJoinReducerBase.reduce(DataJoinReducerBase.java:129)
>         at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:519)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
>
>
>
> It is poiting to  String dataClz = in.readUTF(); this line in readFields
>
>
> *
>
> public
> *
>
> *void* readFields( DataInput in) *throws* IOException
>
> {
>
> *this*.tag.readFields( in);
>
> //String dataClz = in.readUTF();
>
> String dataClz = in.readUTF(); ----> error log show this line is culprit
>
> *try*
> * *
>
> {
>
> //try - catch is needed because the " error: unreported exception
>
> //ClassNotFoundException; must be caught or declared to be thrown"
>
> //is "raised" from compiler
>
> *if*( *this*.data == *null* || !*this*.data.getClass().getName().equals(
> dataClz))
>
> {
>
> //this line of code "raises" the compile error mentioned above
>
> *this*.data = (Writable) ReflectionUtils.*newInstance*( Class.*forName*(
> dataClz), *null*);
>
> }
>
> *this*.data.readFields( in);
>
> }
>
> *catch*( ClassNotFoundException cnfe)
>
> {
>
> System.*out*.println( "Problem in TaggedWritable class, method
> readFields.");
>
> }
>
> }//end readFields
>
>
> --
> *
> *
> *
>
> Thanx and Regards*
> * Vikas Jadhav*
>

--
+Vinod
Hortonworks Inc.
http://hortonworks.com/