Hello,
I am trying to write my very first MapReduce code.  When I try to run the
jar, I get this error:

11/10/15 17:17:30 INFO mapred.JobClient: Task Id :
attempt_201110151636_0003_m_000001_2, Status : FAILED
java.lang.ClassCastException: class edu.bing.vfi5.KeyList
at java.lang.Class.asSubclass(Class.java:3018)
at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:599)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:791)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)

I assume this means that it has something to do with my implementation of
comparable.  KeyList is a class for a 3-tuple key.  The code is listed
below.  Any hints would be greatly appreciated as I am trying to understand
how comparable is supposed to work.  Also, do I need to implement Writable
as well?  If so, should this be code for how the output is written to a file
in HDFS?

Thanks,
Keith

package edu.bing.vfi5;

public class KeyList implements Comparable<KeyList> {

private int[] keys;
 public KeyList(int i, int j, int k) {
keys = new int[3];
keys[0] = i;
keys[0] = j;
keys[0] = k;
}

@Override
public int compareTo(KeyList k) {
// TODO Auto-generated method stub
if(this.keys[0] == k.keys[0] && this.keys[1] == k.keys[1] && this.keys[2] =k.keys[2])
return 0;
else if((this.keys[0]>k.keys[0])
||(this.keys[0]==k.keys[0]&&this.keys[1]>k.keys[1])
||(this.keys[0]==k.keys[0]&&this.keys[1]==k.keys[1]&&this.keys[2]>k.keys[2]))
return 1;
else
return -1;
}
}

Hi,

Discussion, below.

On Sat, Oct 15, 2011 at 4:26 PM, Keith Thompson <[EMAIL PROTECTED]>wrote:

> Hello,
> I am trying to write my very first MapReduce code.  When I try to run the
> jar, I get this error:
>
> 11/10/15 17:17:30 INFO mapred.JobClient: Task Id :
> attempt_201110151636_0003_m_000001_2, Status : FAILED
> java.lang.ClassCastException: class edu.bing.vfi5.KeyList
> at java.lang.Class.asSubclass(Class.java:3018)
> at
> org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:599)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:791)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>
> I assume this means that it has something to do with my implementation of
> comparable.  KeyList is a class for a 3-tuple key.  The code is listed
> below.  Any hints would be greatly appreciated as I am trying to understand
> how comparable is supposed to work.  Also, do I need to implement Writable
> as well?  If so, should this be code for how the output is written to a
> file
> in HDFS?
>
> Thanks,
> Keith
>
> package edu.bing.vfi5;
>
> public class KeyList implements Comparable<KeyList> {
>

Key's need to be WritableComparable.
>
> private int[] keys;
>  public KeyList(int i, int j, int k) {
> keys = new int[3];
> keys[0] = i;
> keys[0] = j;
> keys[0] = k;
> }
>
> @Override
> public int compareTo(KeyList k) {
> // TODO Auto-generated method stub
> if(this.keys[0] == k.keys[0] && this.keys[1] == k.keys[1] && this.keys[2]
> => k.keys[2])
> return 0;
> else if((this.keys[0]>k.keys[0])
> ||(this.keys[0]==k.keys[0]&&this.keys[1]>k.keys[1])
>
> ||(this.keys[0]==k.keys[0]&&this.keys[1]==k.keys[1]&&this.keys[2]>k.keys[2]))
> return 1;
> else
> return -1;
> }
> }
>

Thanks.  I went back and changed to WritableComparable instead of just
Comparable.  So, I added the readFields and write methods.   I also took
care of the typo in the constructor. :P

Now I am getting this error:

11/10/16 21:34:08 INFO mapred.JobClient: Task Id :
attempt_201110162105_0002_m_000001_1, Status : FAILED
java.lang.RuntimeException: java.lang.NoSuchMethodException:
edu.bing.vfi5.KeyList.<init>()
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at
org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:84)
at
org.apache.hadoop.io.WritableComparator.<init>(WritableComparator.java:70)
at org.apache.hadoop.io.WritableComparator.get(WritableComparator.java:44)
at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:599)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:791)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.NoSuchMethodException: edu.bing.vfi5.KeyList.<init>()
at java.lang.Class.getConstructor0(Class.java:2706)
at java.lang.Class.getDeclaredConstructor(Class.java:1985)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)

Is it saying it can't find the constructor?

On Sat, Oct 15, 2011 at 5:26 PM, Keith Thompson <[EMAIL PROTECTED]>wrote:

> Hello,
> I am trying to write my very first MapReduce code.  When I try to run the
> jar, I get this error:
>
> 11/10/15 17:17:30 INFO mapred.JobClient: Task Id :
> attempt_201110151636_0003_m_000001_2, Status : FAILED
> java.lang.ClassCastException: class edu.bing.vfi5.KeyList
> at java.lang.Class.asSubclass(Class.java:3018)
> at
> org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:599)
>  at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:791)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
>  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>
> I assume this means that it has something to do with my implementation of
> comparable.  KeyList is a class for a 3-tuple key.  The code is listed
> below.  Any hints would be greatly appreciated as I am trying to understand
> how comparable is supposed to work.  Also, do I need to implement Writable
> as well?  If so, should this be code for how the output is written to a file
> in HDFS?
>
> Thanks,
> Keith
>
> package edu.bing.vfi5;
>
> public class KeyList implements Comparable<KeyList> {
>
> private int[] keys;
>  public KeyList(int i, int j, int k) {
> keys = new int[3];
>  keys[0] = i;
> keys[1] = j;
> keys[2] = k;
>  }
>
> @Override
> public int compareTo(KeyList k) {
>  // TODO Auto-generated method stub
> if(this.keys[0] == k.keys[0] && this.keys[1] == k.keys[1] && this.keys[2]
> == k.keys[2])
>  return 0;
> else if((this.keys[0]>k.keys[0])
> ||(this.keys[0]==k.keys[0]&&this.keys[1]>k.keys[1])
>
> ||(this.keys[0]==k.keys[0]&&this.keys[1]==k.keys[1]&&this.keys[2]>k.keys[2]))
> return 1;
>  else
> return -1;
> }
> }
>
>
--
*Keith Thompson*
Graduate Research Associate, Xerox Corporation
SUNY Research Foundation
Dept. of Systems Science and Industrial Engineering
Binghamton University
work: 585-422-6587

Hi,

Inline..

On Sun, Oct 16, 2011 at 9:40 PM, Keith Thompson <[EMAIL PROTECTED]>wrote:

> Thanks.  I went back and changed to WritableComparable instead of just
> Comparable.  So, I added the readFields and write methods.   I also took
> care of the typo in the constructor. :P
>
> Now I am getting this error:
>
> 11/10/16 21:34:08 INFO mapred.JobClient: Task Id :
> attempt_201110162105_0002_m_000001_1, Status : FAILED
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> edu.bing.vfi5.KeyList.<init>()
> at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at
> org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:84)
> at
> org.apache.hadoop.io.WritableComparator.<init>(WritableComparator.java:70)
> at org.apache.hadoop.io.WritableComparator.get(WritableComparator.java:44)
> at
> org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:599)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:791)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.NoSuchMethodException: edu.bing.vfi5.KeyList.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
>
> Is it saying it can't find the constructor?
>
>
Writables and by extension WritableComparables need a default Constructor.
This makes logical sense. If hadoop is going to call the readFields()
method, it needs  a previously constructed object.

Brock

Ahh ok ...I think I understand now. I added the default constructor (just
initializing all values to 0) and now it seems to be running.  :-)  Thanks
for your help.

On Sun, Oct 16, 2011 at 9:43 PM, Brock Noland <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Inline..
>
> On Sun, Oct 16, 2011 at 9:40 PM, Keith Thompson <[EMAIL PROTECTED]
> >wrote:
>
> > Thanks.  I went back and changed to WritableComparable instead of just
> > Comparable.  So, I added the readFields and write methods.   I also took
> > care of the typo in the constructor. :P
> >
> > Now I am getting this error:
> >
> > 11/10/16 21:34:08 INFO mapred.JobClient: Task Id :
> > attempt_201110162105_0002_m_000001_1, Status : FAILED
> > java.lang.RuntimeException: java.lang.NoSuchMethodException:
> > edu.bing.vfi5.KeyList.<init>()
> > at
> >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> > at
> >
> org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:84)
> > at
> >
> org.apache.hadoop.io.WritableComparator.<init>(WritableComparator.java:70)
> > at
> org.apache.hadoop.io.WritableComparator.get(WritableComparator.java:44)
> > at
> > org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:599)
> > at
> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:791)
> > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> > at org.apache.hadoop.mapred.Child.main(Child.java:170)
> > Caused by: java.lang.NoSuchMethodException:
> edu.bing.vfi5.KeyList.<init>()
> > at java.lang.Class.getConstructor0(Class.java:2706)
> > at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> > at
> >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
> >
> > Is it saying it can't find the constructor?
> >
> >
> Writables and by extension WritableComparables need a default Constructor.
> This makes logical sense. If hadoop is going to call the readFields()
> method, it needs  a previously constructed object.
>
> Brock
>

--
*Keith Thompson*
Graduate Research Associate, Xerox Corporation
SUNY Research Foundation
Dept. of Systems Science and Industrial Engineering
Binghamton University
work: 585-422-6587