Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to use MapWritable?!


Copy link to this message
-
Re: How to use MapWritable?!
That is what I get in Eclipse's console:

12/02/06 00:50:08 INFO WordCount: emitindo... <perif?rica(1,1)>

12/02/06 00:50:08 INFO WordCount: freq = 3

12/02/06 00:50:08 INFO WordCount: emitindo... <esp?cie(1,3)>

12/02/06 00:50:08 INFO WordCount: freq = 2

12/02/06 00:50:08 INFO WordCount: emitindo... <very(1,2)>

12/02/06 00:50:08 INFO WordCount: key: 1

12/02/06 00:50:08 WARN mapred.LocalJobRunner: job_local_0001

java.util.NoSuchElementException

at java.util.HashMap$HashIterator.nextEntry(HashMap.java:796)

at java.util.HashMap$KeyIterator.next(HashMap.java:828)

at WordCount$MapClass.map(WordCount.java:114)

at WordCount$MapClass.map(WordCount.java:1)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:435)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)

12/02/06 00:50:09 INFO mapred.JobClient: Job complete: job_local_0001
Where WordCount.java:114 is:

LOG.info("freq: " + docIDfreq.get(docIDfreqItr.next()));

If I put values there, why can't I recover them?
Regards,
Luiz

2012/2/5 Luiz Antonio Falaguasta Barbosa <[EMAIL PROTECTED]>

> Hi people,
>
> Please, I don't know what is happening with values I'm putting into a
> MapWritable.
>
> I think I have to call write method (or readFields?) but I don't know how.
> Is it that I need to do to get values through 'LOG.info("freq: " +
> docIDfreq.get(docIDfreqItr.next()));', for example?
>
> Why does it work with key? Look: 'LOG.info("key: " +
> (docIDfreqItr.next()));'
>
> Look the code:
>
>  private static final Log LOG = LogFactory.getLog(WordCount.class
> .getName());
>
>
>  public static class MapClass extends MapReduceBase
>
> implements Mapper<LongWritable, Text, Text, MapWritable> {
>
>
>  public void map(LongWritable key, Text value,
>
>   OutputCollector<Text, MapWritable> output,
>
>   Reporter reporter) throws IOException {
>
>
>   String line = value.toString();
>
>  StringTokenizer itr = new StringTokenizer(line);
>
>  HashMap<String,Integer> map = new HashMap<String,Integer>();
>
>  Integer valor = 0;
>
>
>   while (itr.hasMoreTokens()) {
>
>   try {
>
>   valor =  map.get(itr.nextToken());
>
>
>    if (valor == null) {
>
>    map.put(itr.nextToken(), 1);
>
>   }
>
>   else {
>
>    map.put(itr.nextToken(), valor + 1);
>
>   }
>
>   } catch (NoSuchElementException e) {
>
>   e.printStackTrace();
>
>   }
>
>  }
>
>
>   Iterator<String> mapItr = map.keySet().iterator();
>
>  MapWritable docIDfreq = new MapWritable();
>
>
>   while (mapItr.hasNext()){
>
>
>   String term = mapItr.next();
>
>   Integer freq = map.get(term);
>
>
>   LOG.info("freq = " + freq);
>
>
>   docIDfreq.put(key, new LongWritable(freq.longValue()));
>
>     LOG.info("emitindo... " + "<" + term + "(" + key + "," + freq + ")>");
>
>   output.collect(new Text(term), docIDfreq);
>
>  }
>
>
>   Iterator<Writable> docIDfreqItr = docIDfreq.keySet().iterator();
>
>
>   while (docIDfreqItr.hasNext()){
>
>   LOG.info("key: " + (docIDfreqItr.next()));
>
>   LOG.info("freq: " + docIDfreq.get(docIDfreqItr.next()));
>
>  }
>
>  }
>
> }
>
> Thanks in advance!
>
> Regards,
>
> Luiz
>

--
[]s,

Luiz
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB