Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Starting Map Reduce Job on EC2


Copy link to this message
-
Re: Starting Map Reduce Job on EC2
Can you open all of the monitoring browser ports?

On Sun, Jan 15, 2012 at 3:03 PM, Something Something
<[EMAIL PROTECTED]> wrote:
> Good point.  Those ports may not be open.  So next question - is it safe to
> open these ports?  How do we securely open these ports to avoid malicious
> attacks under EC2?
>
> (Sorry, I know some of these questions are dumb - but we are a startup and
> don't have a big sysadmin group - I guess that's why we are using EC2 -:)
>
>
> On Sun, Jan 15, 2012 at 12:03 PM, Ronald Petty <[EMAIL PROTECTED]>
> wrote:
>>
>> Something Something,
>>
>> Have you confirmed you can connect to the port from your remote machine?
>>
>> telnet ec2-xxxxxx.... 9000
>>
>> Kindest regards.
>>
>> Ron
>>
>> On Sun, Jan 15, 2012 at 12:16 AM, Something Something
>> <[EMAIL PROTECTED]> wrote:
>>>
>>> Hello,
>>>
>>> Our Hadoop cluster is setup on EC2, but our client machine which will
>>> trigger the M/R job is in our data center.  I am trying to start a M/R job
>>> from our client machine, but getting this:
>>>
>>> 00:01:16.885 [pool-6-thread-1] INFO  org.apache.hadoop.ipc.Client -
>>> Retrying connect to server:
>>> ec2-xx-xx-xxx-xxx.compute-1.amazonaws.com/xx.xx.xxx.xx:9000. Already tried 0
>>> time(s).
>>> 00:01:16.885 [pool-6-thread-1] INFO  org.apache.hadoop.ipc.Client -
>>> Retrying connect to server:
>>> ec2-xx-xx-xxx-xxx.compute-1.amazonaws.com/xx.xx.xxx.xx:9000. Already tried 1
>>> time(s).
>>> 00:01:16.885 [pool-6-thread-1] INFO  org.apache.hadoop.ipc.Client -
>>> Retrying connect to server:
>>> ec2-xx-xx-xxx-xxx.compute-1.amazonaws.com/xx.xx.xxx.xx:9000. Already tried 2
>>> time(s).
>>> & so on...
>>>
>>>
>>> I tried putting our 'pem' file in the classpath, but that didn't help.  I
>>> am assuming it should be possible to trigger a job on EC2 cluster from a
>>> machine that's not in EC2, correct?
>>>
>>> Please note that I can trigger a M/R job from the client machine to a
>>> Hadoop running on the same machine - localhost:9000 (hdfs) as well as
>>> localhost:9001 (jobtracker).
>>>
>>> Please help.  Thanks.
>>>
>>>
>>>
>>>
>>
>

--
Lance Norskog
[EMAIL PROTECTED]
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB