Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> mrunit question


Copy link to this message
-
Re: mrunit question
On 08/11/2010 08:14 PM, Aaron Kimball wrote:
> David,
>
> Since you are directly instantiating the Mapper and Reducer (not using
> ReflectionUtils), you are free to call setConf() yourself before you run the
> test.

Sort of.  What would wind up happening is that setConf would get called
twice:  once by the mrunit framework with the empty config (for which
I'd need to add special coding in to my reducer object to make it ignore
it) and then a second time when I explicitly call it myself in my JUnit
test.  It's not really ideal for me to have to add special code into my
reducer to avoid this issue.
> If you're using the new API, there already is setConfiguration() and
> withConfiguration(), which will set the Configuration being passed into
> Context sent to your Mapper/Reducer's run() method.
> See https://issues.apache.org/jira/browse/mapreduce-1569 where this was
> introduced.
>
>
> - Aaron

Yes, just looked at that, and it looks like it exactly solves this
issue.  Unfortunately, though, it says that fix is effective in v0.21.
As we're running the stable CDH version (CDH2, which uses Hadoop v0.20)
I can't take advantage of that fix yet.

Going to wind up forgoing unit tests on this M/R code for right now I think.

Thanks,

DR
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB