MapReduce, mail # user - who runs the map and reduce tasks in the unit tests

who runs the map and reduce tasks in the unit tests

Pedro Sá da Costa 2013-02-21, 20:11

Hi,

In Hadoop MR unit tests, the classes uses the
./core/org/apache/hadoop/util/Tool.java, and
./core/org/apache/hadoop/util/ToolRunner.java tosubmit the job. But to
run the unit tests it seems that it's not needed the MR be running. If
so, who runs the map and reduce tasks?
--
Best regards,
P

Re: who runs the map and reduce tasks in the unit tests

Vinod Kumar Vavilapalli 2013-02-21, 21:07

Not sure which specific tests you are talking about. There are two types of them:
- Real unit tests which unit test code, shouldn't run any MR jobs
- The remaining 'unit' tests are really integration tests. They start MiniMRCluster and MiniDFSCluster (which are basically in-JVM MR and DFS) and submit jobs to them.

HTH

+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Feb 21, 2013, at 12:11 PM, Pedro Sá da Costa wrote:

> Hi,
>
> In Hadoop MR unit tests, the classes uses the
> ./core/org/apache/hadoop/util/Tool.java, and
> ./core/org/apache/hadoop/util/ToolRunner.java tosubmit the job. But to
> run the unit tests it seems that it's not needed the MR be running. If
> so, who runs the map and reduce tasks?
>
>
> --
> Best regards,
> P