Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Secondary Sort example error


Copy link to this message
-
Re: Secondary Sort example error
Hi,

I am still not able to find the fix for this issue.  have tried to change
the attribute datatypes from String to Text.. But I still have the same
error.. Is there anything else that I can try?

Thanks
On Fri, Feb 8, 2013 at 12:30 AM, Ravi Chandran <[EMAIL PROTECTED]>wrote:

> Thanks for replying.  . I believe writable and writablecomparable handle
> the serialization..  I found the required class
>
> public class Name implements WritableComparable<Name>{
>
>  private String fName;
>  private String lName;
>   static {            // register this comparator
>        WritableComparator.define(Name.class, new NameSorterComparator());
>      }
>
>
>  public Name()
>  {
>
>  }
>
>  public Name(String first, String last)
>  {
>   set(first, last);
>  }
>
>   public void set(String first, String last) {
>    this.fName = first;
>   this.lName = last;
>   }
>  public String getfName() {
>   return fName;
>  }
>  public void setfName(String fName) {
>   this.fName = fName;
>  }
>  public String getlName() {
>   return lName;
>  }
>  public void setlName(String lName) {
>   this.lName = lName;
>  }
>  public String toString()
>  {
>   return this.getfName()+" "+this.getlName();
>  }
>
>  @Override
>  public void write(DataOutput out) throws IOException {
>   // TODO Auto-generated method stub
>   out.writeUTF(fName);
>   out.writeUTF(lName);
>  }
>  public boolean equals(Name  o) {
>   Name other = o;
>   if(this.fName.toString().equalsIgnoreCase(other.fName.toString()))
>   {
>    if(this.lName.toString().equalsIgnoreCase(other.lName.toString()))
>    {
>     return true;
>    }
>   }
>   return false;
>  }
>  @Override
>  public void readFields(DataInput in) throws IOException {
>   // TODO Auto-generated method stub
>   this.fName = in.readUTF();
>   this.lName = in.readUTF();
>  }
>  @Override
>  public int hashCode() {
>   return fName.hashCode() * 514 + lName.hashCode();
>  }
>  @Override
>  public int compareTo(Name tp) {
>   int cmp = fName.compareTo(tp.fName);
>   if (cmp != 0) {
>    return cmp;
>   }
>   return lName.compareTo(tp.lName);
>  }
> }
>
>
> On Fri, Feb 8, 2013 at 12:16 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Thanks, I managed to correlate proper line numbers.
>>
>> Are you using some form of custom serialization in your job code? That
>> is, are your keys non-Writable types and are of some other type? The
>> specific NPE is arising from the SerializationFactory not being able
>> to find a serializer for your Map-Output key class. You may want to
>> look into that direction, or share your code for the list to spot it
>> instead.
>>
>> On Fri, Feb 8, 2013 at 12:11 AM, Ravi Chandran
>> <[EMAIL PROTECTED]> wrote:
>> > hi,
>> >
>> > it is Hadoop 2.0.0-cdh4.1.1. the whole output is given below:
>> >
>> > Hadoop 2.0.0-cdh4.1.1
>> > Subversion
>> >
>> file:///data/1/jenkins/workspace/generic-package-centos32-6/topdir/BUILD/hadoop-2.0.0-cdh4.1.1/src/hadoop-common-project/hadoop-common
>> > -r 581959ba23e4af85afd8db98b7687662fe9c5f20
>> >
>> >
>> >
>> > On Fri, Feb 8, 2013 at 12:04 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>> >>
>> >> Hey Ravi,
>> >>
>> >> What version of Hadoop is this exactly? (Type and send output of
>> >> "hadoop version" if unsure)
>> >>
>> >> On Thu, Feb 7, 2013 at 11:55 PM, Ravi Chandran
>> >> <[EMAIL PROTECTED]> wrote:
>> >> > Hi,
>> >> >
>> >> > I am trying to do a name sorting using secondary sort. I have a
>> working
>> >> > example, which I am taking as a reference. But I am getting a null
>> >> > pointer
>> >> > error in the MapTask class. I am not able to locate the reason. as
>> the
>> >> > logic
>> >> > to create the Custom Object from a given file has been tested
>> through a
>> >> > java
>> >> > class..
>> >> > I am getting this error:
>> >> >
>> >> > 13/02/07 12:23:42 WARN snappy.LoadSnappy: Snappy native library is
>> >> > available
>> >> > 13/02/07 12:23:42 INFO snappy.LoadSnappy: Snappy native library
>> loaded
>> >> > 13/02/07 12:23:42 INFO mapred.FileInputFormat: Total input paths to

Thanks & Regards
Ravi