Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Secondary Sort example error


Copy link to this message
-
Secondary Sort example error
Hi,

I am trying to do a name sorting using secondary sort. I have a working
example, which I am taking as a reference. But I am getting a null pointer
error in the MapTask class. I am not able to locate the reason. as the
logic to create the Custom Object from a given file has been tested through
a java class..
I am getting this error:

13/02/07 12:23:42 WARN snappy.LoadSnappy: Snappy native library is available
13/02/07 12:23:42 INFO snappy.LoadSnappy: Snappy native library loaded
13/02/07 12:23:42 INFO mapred.FileInputFormat: Total input paths to process
: 1
13/02/07 12:23:43 INFO mapred.JobClient: Running job: job_201301301056_0014
13/02/07 12:23:44 INFO mapred.JobClient:  map 0% reduce 0%
13/02/07 12:23:56 INFO mapred.JobClient: Task Id :
attempt_201301301056_0014_m_000000_0, Status : FAILED
java.lang.NullPointerException
 at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:814)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:385)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
 at org.apache.hadoop.mapred.Child.main(Child.java:262)
13/02/07 12:23:57 INFO mapred.JobClient: Task Id :
attempt_201301301056_0014_m_000001_0, Status : FAILED
java.lang.NullPointerException
 at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:814)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:385)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
 at org.apache.hadoop.mapred.Child.main(Child.java:262)

I am giving the Mapper code below:

import java.io.IOException;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;
import org.apache.log4j.Logger;
import com.pom.Name;

public class StubMapper extends MapReduceBase implements
    Mapper<LongWritable, Text, Name, Text> {

  private static Logger logger     Logger.getLogger(StubMapper.class.getName());

 StringBuffer readLine = new StringBuffer();
  private final Name name = new Name();
  @Override
  public void map(LongWritable key, Text value,
      OutputCollector<Name, Text> output, Reporter reporter)
      throws IOException {
    String line = value.toString();
    String[] packer = null;

    packer = line.split(" ");

    // create the object
 if(packer.length>2)
 {
  // take everything except last name
  for (int i = 0; i < packer.length-1; i++) {
   readLine.append(packer[i]+" ");
  }

  name.setfName(readLine.toString());
  name.setlName(packer[packer.length-1]);

  //clear the variable
  readLine.delete(0, readLine.length());
 }
 else if(packer.length>0)
 {
  name.setfName(packer[0]);
     name.setlName(packer[1]);
   }

 output.collect(name, new Text(name.getlName()));

  }
}

I am not able to figure out the possible cause..

--
Thanks & Regards
Ravi
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB