Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Issue with Reduce Side join using datajoin package


Copy link to this message
-
Re: Issue with Reduce Side join using datajoin package
Seems like a bug in your code, can you share the source here?

+Vinod
On Tue, Jan 29, 2013 at 4:00 AM, Vikas Jadhav <[EMAIL PROTECTED]>wrote:

> I am using Hadoop 1.0.3
>
> I am getting following Error
>
>
> 13/01/29 06:55:19 INFO mapred.JobClient: Task Id :
> attempt_201301290120_0006_r_000000_0, Status : FAILED
> java.lang.NullPointerException
>         at MyJoin$TaggedWritable.readFields(MyJoin.java:101)
>         at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
>         at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
>         at
> org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:1271)
>         at
> org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:1211)
>         at
> org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:249)
>         at
> org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:245)
>         at
> org.apache.hadoop.contrib.utils.join.DataJoinReducerBase.regroup(DataJoinReducerBase.java:106)
>         at
> org.apache.hadoop.contrib.utils.join.DataJoinReducerBase.reduce(DataJoinReducerBase.java:129)
>         at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:519)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
>
>
>
> It is poiting to  String dataClz = in.readUTF(); this line in readFields
>
>
> *
>
> public
> *
>
> *void* readFields( DataInput in) *throws* IOException
>
> {
>
> *this*.tag.readFields( in);
>
> //String dataClz = in.readUTF();
>
> String dataClz = in.readUTF(); ----> error log show this line is culprit
>
> *try*
> * *
>
> {
>
> //try - catch is needed because the " error: unreported exception
>
> //ClassNotFoundException; must be caught or declared to be thrown"
>
> //is "raised" from compiler
>
> *if*( *this*.data == *null* || !*this*.data.getClass().getName().equals(
> dataClz))
>
> {
>
> //this line of code "raises" the compile error mentioned above
>
> *this*.data = (Writable) ReflectionUtils.*newInstance*( Class.*forName*(
> dataClz), *null*);
>
> }
>
> *this*.data.readFields( in);
>
> }
>
> *catch*( ClassNotFoundException cnfe)
>
> {
>
> System.*out*.println( "Problem in TaggedWritable class, method
> readFields.");
>
> }
>
> }//end readFields
>
>
> --
> *
> *
> *
>
> Thanx and Regards*
> * Vikas Jadhav*
>

--
+Vinod
Hortonworks Inc.
http://hortonworks.com/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB