Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Re: good way to debug map reduce code


Copy link to this message
-
Re: good way to debug map reduce code
Rishi Yadav 2012-12-26, 06:48
pseudo-distributed mode is the only way you can test your code as far as I
know. This means that you are running a single node cluster. Are you using
eclipse?

Thanks and Regards,

Rishi Yadav

(o) 408.988.2000x113 ||  (f) 408.716.2726

InfoObjects Inc || http://www.infoobjects.com *(Big Data Solutions)*

*INC 500 Fastest growing company in 2012 || 2011*

*Best Place to work in Bay Area 2012 - *SF Business Times and the Silicon
Valley / San Jose Business Journal

2041 Mission College Boulevard, #280 || Santa Clara, CA 95054
On Tue, Dec 25, 2012 at 5:56 PM, jamal sasha <[EMAIL PROTECTED]> wrote:

> Hi,
>   I have been using python hadoop streaming framework to write the code
> and now I am slowly moving towards the core java api's.
> And I am getting comfortable with it but what is the quickest way to debug
> the map reduce native code..
> like in hadoop streaming this worked great.
> % cat input.txt | python mapper.py | sort | python reducer.py
>
> If there use to be any coding error.. it use to just throw them off and it
> was very fast to debug as you code.
> Is there any similar way .. where i dont have to run hadoop jobs to debg
> and wait and go thru hadoop logs to see that maybe i miss a semi-colon..
> Thanks
> Jamal
>