Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Starting Map Reduce Job on EC2


Copy link to this message
-
Re: Starting Map Reduce Job on EC2
Lance Norskog 2012-01-16, 01:00
Can you open all of the monitoring browser ports?

On Sun, Jan 15, 2012 at 3:03 PM, Something Something
<[EMAIL PROTECTED]> wrote:
> Good point.  Those ports may not be open.  So next question - is it safe to
> open these ports?  How do we securely open these ports to avoid malicious
> attacks under EC2?
>
> (Sorry, I know some of these questions are dumb - but we are a startup and
> don't have a big sysadmin group - I guess that's why we are using EC2 -:)
>
>
> On Sun, Jan 15, 2012 at 12:03 PM, Ronald Petty <[EMAIL PROTECTED]>
> wrote:
>>
>> Something Something,
>>
>> Have you confirmed you can connect to the port from your remote machine?
>>
>> telnet ec2-xxxxxx.... 9000
>>
>> Kindest regards.
>>
>> Ron
>>
>> On Sun, Jan 15, 2012 at 12:16 AM, Something Something
>> <[EMAIL PROTECTED]> wrote:
>>>
>>> Hello,
>>>
>>> Our Hadoop cluster is setup on EC2, but our client machine which will
>>> trigger the M/R job is in our data center.  I am trying to start a M/R job
>>> from our client machine, but getting this:
>>>
>>> 00:01:16.885 [pool-6-thread-1] INFO  org.apache.hadoop.ipc.Client -
>>> Retrying connect to server:
>>> ec2-xx-xx-xxx-xxx.compute-1.amazonaws.com/xx.xx.xxx.xx:9000. Already tried 0
>>> time(s).
>>> 00:01:16.885 [pool-6-thread-1] INFO  org.apache.hadoop.ipc.Client -
>>> Retrying connect to server:
>>> ec2-xx-xx-xxx-xxx.compute-1.amazonaws.com/xx.xx.xxx.xx:9000. Already tried 1
>>> time(s).
>>> 00:01:16.885 [pool-6-thread-1] INFO  org.apache.hadoop.ipc.Client -
>>> Retrying connect to server:
>>> ec2-xx-xx-xxx-xxx.compute-1.amazonaws.com/xx.xx.xxx.xx:9000. Already tried 2
>>> time(s).
>>> & so on...
>>>
>>>
>>> I tried putting our 'pem' file in the classpath, but that didn't help.  I
>>> am assuming it should be possible to trigger a job on EC2 cluster from a
>>> machine that's not in EC2, correct?
>>>
>>> Please note that I can trigger a M/R job from the client machine to a
>>> Hadoop running on the same machine - localhost:9000 (hdfs) as well as
>>> localhost:9001 (jobtracker).
>>>
>>> Please help.  Thanks.
>>>
>>>
>>>
>>>
>>
>

--
Lance Norskog
[EMAIL PROTECTED]