Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to connect to a cluster by using eclipse


Copy link to this message
-
Re: How to connect to a cluster by using eclipse
Jason,
Ramon is right.
The best way to debug a MapReduce job is mounting a local cluster, and
then, when you have tested enough your code, then, you can
deploy it in a real distributed cluster.
On 07/04/2012 10:00 PM, Jason Yang wrote:
> ramon,
>
> Thank for your reply very much.
>
> However, I was still wonder whether I could debug a MR application in
> this way.
>
> I have read some posts talking about using NAT to redirect all the
> packets to the network card which connect to the local LAN, but it
> does not work as I tried to redirect by using iptables :(
>
> 锟斤拷 2012锟斤拷7锟斤拷4锟斤拷锟斤拷锟斤拷锟斤拷 写锟斤拷锟斤拷
>
>     Jason,
>
>
>         the easiest way to debug a MapRedupe program with eclipse is
>         working on hadoop local.
>         http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#Local
>
>
>         In this mode all the components run locally on the same VM and
>         can be easily debugged using Eclipse.
>
>
>         Hope this will be useful.
>
>     *From:*Jason Yang [mailto:[EMAIL PROTECTED]
>     <javascript:_e({}, 'cvml', '[EMAIL PROTECTED]');>]
>     *Sent:* mi锟斤拷rcoles, 04 de julio de 2012 11:25
>     *To:* mapreduce-user
>     *Subject:* How to connect to a cluster by using eclipse
>
>     Hi, all
>
>     I have a hadoop cluster with 3 nodes, the network topology is like
>     this:
>
>     1. For each DataNode, its IP address is like :192.168.0.XXX;
>
>     2. For the NameNode, it has two network cards: one is connect with
>     the DataNodes as a local LAN with IP address 192.168.0.110, while
>     the other one is connect to the company network(which eventually
>     connect to the Internet);
>
>     --
>
>     now I'm trying to debug a MapReduce program on a computer which is
>     in the company network. Since the jobtracker in this scenario is
>     192.168.0.110:9001 <http://192.168.0.110:9001>, I was wondering
>     how could I connect to the cluster by using eclipse?
>
>     --
>
>     YANG, Lin
>
>
>     ------------------------------------------------------------------------
>     Subject to local law, communications with Accenture and its
>     affiliates including telephone calls and emails (including
>     content), may be monitored by our systems for the purposes of
>     security and the assessment of internal compliance with Accenture
>     policy.
>     ______________________________________________________________________________________
>
>     www.accenture.com <http://www.accenture.com>
>
>
>
> --
> YANG, Lin
>

--

Marcos Luis Ort锟斤拷z Valmaseda
*Data Engineer && Sr. System Administrator at UCI*
about.me/marcosortiz <http://about.me/marcosortiz>
My Blog <http://marcosluis2186.posterous.com>
@marcosluis2186 <http://twitter.com/marcosluis2186>

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci