Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> MultithreadedMapper in 0.20.0


Copy link to this message
-
Re: MultithreadedMapper in 0.20.0
MultithreadedMapper in 0.20.0 has got bug. It got fixed in 0.21 by http://issues.apache.org/jira/browse/MAPREDUCE-465

Thanks
Amareshwari

Mathias De Maré wrote:
> Hi,
>
> I was wondering if MultithreadedMapper is broken in 0.20.0, or perhaps I am
> using it incorrectly.
> The following code:
>
> Configuration conf = new Configuration();
> String[] otherArgs = new GenericOptionsParser(conf,
> args).getRemainingArgs();
> Job job = new Job(conf, "foo");
> job.setMapperClass(MultithreadedMapper.class);
> MultithreadedMapper.setMapperClass(job, CrawlerMapper.class);
>
> The last line always fails with the following error:
>     [javac] Compiling 14 source files to
> /home/mathias/hadoop_testing/crawler/svn/Crawler/build/classes
>     [javac]
> /home/mathias/hadoop_testing/crawler/svn/Crawler/src/mycrawler/CrawlerJob.java:37:
> <K1,V1,K2,V2>setMapperClass(org.apache.hadoop.mapreduce.Job,java.lang.Class<org.apache.hadoop.mapreduce.Mapper<K1,V1,K2,V2>>)
> in org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper cannot be applied
> to
> (com.acquia.web.crawler.CrawlerJob,java.lang.Class<mycrawler.CrawlerMapper>)
>     [javac]         MultithreadedMapper.setMapperClass(this,
> CrawlerMapper.class);
>     [javac]                            ^
>     [javac] 1 error
>
> However, CrawlerMapper does extend Mapper! More precisely: public class
> CrawlerMapper extends Mapper<IntWritable, URLInfo, Text, URLInfo>
>
> Is there perhaps something I'm missing?
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB