Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> error:Type mismatch in value from map


Copy link to this message
-
Re: error:Type mismatch in value from map
If you want to use a combiner, your map has to output the same types
as your combiner outputs. In your case, modify your map to look like
this:

  public static class TokenizerMapper
       extends Mapper<Text, Text, Text, IntWritable>{
    public void map(Text key, Text value, Context context
                    ) throws IOException, InterruptedException {
        context.write(key, new IntWritable(1));
    }
  }

>  11/07/29 22:22:22 INFO mapred.JobClient: Task Id : attempt_201107292131_0011_m_000000_2, Status : FAILED
> java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.IntWritable, recieved org.apache.hadoop.io.Text
>
> But I already set IntWritable in 2 places,
> 1: Reducer<Text,Text,Text,IntWritable>
> 2:job.setOutputValueClass(IntWritable.class);
>
> So where am I wrong?
>
> public class MyTest {
>
>  public static class TokenizerMapper
>       extends Mapper<Text, Text, Text, Text>{
>    public void map(Text key, Text value, Context context
>                    ) throws IOException, InterruptedException {
>        context.write(key, value);
>    }
>  }
>
>  public static class IntSumReducer
>       extends Reducer<Text,Text,Text,IntWritable> {
>
>    public void reduce(Text key, Iterable<Text> values,
>                       Context context
>                       ) throws IOException, InterruptedException {
>       int count = 0;
>       for (Text iw:values) {
>            count++;
>       }
>      context.write(key, new IntWritable(count));
>     }
>   }
>
>  public static void main(String[] args) throws Exception {
>    Configuration conf = new Configuration();
> // the configure of seprator should be done in conf
>    conf.set("key.value.separator.in.input.line", ",");
>    String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();
>    if (otherArgs.length != 2) {
>      System.err.println("Usage: wordcount <in> <out>");
>      System.exit(2);
>    }
>    Job job = new Job(conf, "word count");
>    job.setJarByClass(WordCount.class);
>    job.setMapperClass(TokenizerMapper.class);
>    job.setCombinerClass(IntSumReducer.class);
> //    job.setReducerClass(IntSumReducer.class);
>    job.setInputFormatClass(KeyValueTextInputFormat.class);
>    // job.set("key.value.separator.in.input.line", ",");
>    job.setOutputKeyClass(Text.class);
>    job.setOutputValueClass(IntWritable.class);
>    FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
>    FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
>    System.exit(job.waitForCompletion(true) ? 0 : 1);
>  }
> }
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB