Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to connect to a cluster by using eclipse


Copy link to this message
-
Re: How to connect to a cluster by using eclipse
Jason,
Ramon is right.
The best way to debug a MapReduce job is mounting a local cluster, and
then, when you have tested enough your code, then, you can
deploy it in a real distributed cluster.
On 07/04/2012 10:00 PM, Jason Yang wrote:
> ramon,
>
> Thank for your reply very much.
>
> However, I was still wonder whether I could debug a MR application in
> this way.
>
> I have read some posts talking about using NAT to redirect all the
> packets to the network card which connect to the local LAN, but it
> does not work as I tried to redirect by using iptables :(
>
> 锟斤拷 2012锟斤拷7锟斤拷4锟斤拷锟斤拷锟斤拷锟斤拷 写锟斤拷锟斤拷
>
>     Jason,
>
>
>         the easiest way to debug a MapRedupe program with eclipse is
>         working on hadoop local.
>         http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#Local
>
>
>         In this mode all the components run locally on the same VM and
>         can be easily debugged using Eclipse.
>
>
>         Hope this will be useful.
>
>     *From:*Jason Yang [mailto:[EMAIL PROTECTED]
>     <javascript:_e({}, 'cvml', '[EMAIL PROTECTED]');>]
>     *Sent:* mi锟斤拷rcoles, 04 de julio de 2012 11:25
>     *To:* mapreduce-user
>     *Subject:* How to connect to a cluster by using eclipse
>
>     Hi, all
>
>     I have a hadoop cluster with 3 nodes, the network topology is like
>     this:
>
>     1. For each DataNode, its IP address is like :192.168.0.XXX;
>
>     2. For the NameNode, it has two network cards: one is connect with
>     the DataNodes as a local LAN with IP address 192.168.0.110, while
>     the other one is connect to the company network(which eventually
>     connect to the Internet);
>
>     --
>
>     now I'm trying to debug a MapReduce program on a computer which is
>     in the company network. Since the jobtracker in this scenario is
>     192.168.0.110:9001 <http://192.168.0.110:9001>, I was wondering
>     how could I connect to the cluster by using eclipse?
>
>     --
>
>     YANG, Lin
>
>
>     ------------------------------------------------------------------------
>     Subject to local law, communications with Accenture and its
>     affiliates including telephone calls and emails (including
>     content), may be monitored by our systems for the purposes of
>     security and the assessment of internal compliance with Accenture
>     policy.
>     ______________________________________________________________________________________
>
>     www.accenture.com <http://www.accenture.com>
>
>
>
> --
> YANG, Lin
>

--

Marcos Luis Ort锟斤拷z Valmaseda
*Data Engineer && Sr. System Administrator at UCI*
about.me/marcosortiz <http://about.me/marcosortiz>
My Blog <http://marcosluis2186.posterous.com>
@marcosluis2186 <http://twitter.com/marcosluis2186>

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB