Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: good way to debug map reduce code


+
SUJIT PAL 2012-12-26, 16:53
Copy link to this message
-
Re: good way to debug map reduce code
For Java MR jobs, there is Apache MRUnit that provides a good way of
writing test cases. See http://mrunit.apache.org

On Wed, Dec 26, 2012 at 7:26 AM, jamal sasha <[EMAIL PROTECTED]> wrote:
> Hi,
>   I have been using python hadoop streaming framework to write the code and
> now I am slowly moving towards the core java api's.
> And I am getting comfortable with it but what is the quickest way to debug
> the map reduce native code..
> like in hadoop streaming this worked great.
> % cat input.txt | python mapper.py | sort | python reducer.py
>
> If there use to be any coding error.. it use to just throw them off and it
> was very fast to debug as you code.
> Is there any similar way .. where i dont have to run hadoop jobs to debg and
> wait and go thru hadoop logs to see that maybe i miss a semi-colon..
> Thanks
> Jamal

--
Harsh J