Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> How to use MapWritable?!


Copy link to this message
-
How to use MapWritable?!
Hi people,

Please, I don't know what is happening with values I'm putting into a
MapWritable.

I think I have to call write method (or readFields?) but I don't know how.
Is it that I need to do to get values through 'LOG.info("freq: " +
docIDfreq.get(docIDfreqItr.next()));', for example?

Why does it work with key? Look: 'LOG.info("key: " +
(docIDfreqItr.next()));'

Look the code:

 private static final Log LOG = LogFactory.getLog(WordCount.class
.getName());
 public static class MapClass extends MapReduceBase

implements Mapper<LongWritable, Text, Text, MapWritable> {
 public void map(LongWritable key, Text value,

  OutputCollector<Text, MapWritable> output,

  Reporter reporter) throws IOException {
  String line = value.toString();

 StringTokenizer itr = new StringTokenizer(line);

 HashMap<String,Integer> map = new HashMap<String,Integer>();

 Integer valor = 0;
  while (itr.hasMoreTokens()) {

  try {

  valor =  map.get(itr.nextToken());
   if (valor == null) {

   map.put(itr.nextToken(), 1);

  }

  else {

   map.put(itr.nextToken(), valor + 1);

  }

  } catch (NoSuchElementException e) {

  e.printStackTrace();

  }

 }
  Iterator<String> mapItr = map.keySet().iterator();

 MapWritable docIDfreq = new MapWritable();
  while (mapItr.hasNext()){
  String term = mapItr.next();

  Integer freq = map.get(term);
  LOG.info("freq = " + freq);
  docIDfreq.put(key, new LongWritable(freq.longValue()));

    LOG.info("emitindo... " + "<" + term + "(" + key + "," + freq + ")>");

  output.collect(new Text(term), docIDfreq);

 }
  Iterator<Writable> docIDfreqItr = docIDfreq.keySet().iterator();
  while (docIDfreqItr.hasNext()){

  LOG.info("key: " + (docIDfreqItr.next()));

  LOG.info("freq: " + docIDfreq.get(docIDfreqItr.next()));

 }

 }

}

Thanks in advance!

Regards,

Luiz
+
Luiz Antonio Falaguasta B... 2012-02-06, 02:53