In writing a Map/Reduce job I ran across something I found a little
strange.  I have a situation where I don't need a value output from map.
If I set the value of the value of OutputCollector<Text, IntWritable> to
null I get the following exception:

 

java.lang.NullPointerException

   at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:56
2)

 

    Looking at the code in MapTask.java ( Hadoop .19.1 ) it makes sense
why it would throw the exception:

 

      if (value.getClass() != valClass) {

        throw new IOException("Type mismatch in value from map: expected
"

                              + valClass.getName() + ", recieved "

                              + value.getClass().getName());

      }

 

  I guess my question is as follows: is it a bad idea/not normal to
collect a null value in map?  Outputting from reduce through
TextOutputFormat with a null value as I expect.  If the value is null
only they key and newline are output.  

 

   Any thoughts would be appreciated.

  

 

  

You can initialize IntWritable with an empty constructor.
IntWritable i=new IntWritable();

On Fri, Mar 13, 2009 at 2:21 PM, Andy Sautins
<[EMAIL PROTECTED]>wrote:

>
>
>   In writing a Map/Reduce job I ran across something I found a little
> strange.  I have a situation where I don't need a value output from map.
> If I set the value of the value of OutputCollector<Text, IntWritable> to
> null I get the following exception:
>
>
>
> java.lang.NullPointerException
>
>   at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:56
> 2)
>
>
>
>    Looking at the code in MapTask.java ( Hadoop .19.1 ) it makes sense
> why it would throw the exception:
>
>
>
>      if (value.getClass() != valClass) {
>
>        throw new IOException("Type mismatch in value from map: expected
> "
>
>                              + valClass.getName() + ", recieved "
>
>                              + value.getClass().getName());
>
>      }
>
>
>
>  I guess my question is as follows: is it a bad idea/not normal to
> collect a null value in map?  Outputting from reduce through
> TextOutputFormat with a null value as I expect.  If the value is null
> only they key and newline are output.
>
>
>
>   Any thoughts would be appreciated.
>
>
>
>
>
>
>
>
--
Richa Khandelwal
University Of California,
Santa Cruz.
Ph:425-241-7763

On Mar 13, 2009, at 3:56 PM, Richa Khandelwal wrote:

> You can initialize IntWritable with an empty constructor.
> IntWritable i=new IntWritable();

NullWritable is better for that application than IntWritable. It  
doesn't consume any space when serialized. *smile*

-- Owen