Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HWI use on AWS/EMR


Copy link to this message
-
Re: HWI use on AWS/EMR
ok so much discussion around this but why dont you open access to port 9999
for your specific machine (home/office) and access it normally without
setting up port forwarding.

AWS does give you ip level access control in the security groups
On Fri, Jan 18, 2013 at 10:35 PM, Ariel Marcus <[EMAIL PROTECTED]>wrote:

> This should be localhost instead of proxy:
>
> - In Connection/SSH/Tunnels I’ve added 9999 in the Source Port box, *
> localhost*:9999 in the Destination box. Not sure if this is correct.
>
> Best,
> Aril
>
> ---------------------------------
> Ariel Marcus, Consultant
> www.openbi.com | [EMAIL PROTECTED]
> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
> Cell: 314-827-4356
>
>
> On Fri, Jan 18, 2013 at 12:00 PM, Tony Burton <[EMAIL PROTECTED]>wrote:
>
>> ** **
>>
>> Ok, thanks Ariel (and Dean in the next email!)****
>>
>> ** **
>>
>> I’ve set up PuTTY (v0.62) as follows:****
>>
>> - EMR instance hostname in the Session/Host Name box (port left at 22,
>> ok?)****
>>
>> - In Connection/SSH/Auth I’ve added my private key file, downloaded from
>> AWS ages for this account.****
>>
>> - In Connection/SSH/Tunnels I’ve added 9999 in the Source Port box,
>> proxy:9999 in the Destination box. Not sure if this is correct.****
>>
>> ** **
>>
>> Hit the Open button, I can connect. (And I can see my changes in
>> hive-default.xml, so I know I’m in the right place:)****
>>
>> Then pointing my browser at http://localhost:9999 I get the same timeout
>> again. Have I set up everything right?****
>>
>> ** **
>>
>> Also, following good advice I copied the config changes to hive-site.xml.
>> How do I stop hive so I can restart with ‘hive –service hwi’ so that it
>> picks up the new config?****
>>
>> ** **
>>
>> Tony****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> *From:* Ariel Marcus [mailto:[EMAIL PROTECTED]]
>> *Sent:* 18 January 2013 16:31
>>
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: HWI use on AWS/EMR****
>>
>> ** **
>>
>> Hey Tony,****
>>
>> ** **
>>
>> Port forwarding is the same as what Dean referred to as ssh tunneling.***
>> *
>>
>> ** **
>>
>> Here is a website explaining how to set up port forwarding in putty (use
>> port 9999):****
>>
>> http://www.cs.uu.nl/technical/services/ssh/putty/puttyfw.html****
>>
>> ** **
>>
>> You can edit either hive-default or hive-site but the general practice is
>> to store changes from the default configuration in hive-site.****
>>
>> ** **
>>
>> Best,****
>>
>> Ariel****
>>
>> ** **
>>
>>
>> ****
>>
>> ---------------------------------****
>>
>> Ariel Marcus, Consultant****
>>
>> www.openbi.com | [EMAIL PROTECTED]****
>>
>> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
>> Cell: 314-827-4356****
>>
>> ** **
>>
>> On Fri, Jan 18, 2013 at 11:26 AM, Tony Burton <[EMAIL PROTECTED]>
>> wrote:****
>>
>> Hi Ariel,****
>>
>>  ****
>>
>> Thanks for the speedy reply. We’ll be accessing the HWI from Windows
>> rather Linux desktops, so can you help me out with understanding how to
>> carry out the port forwarding from a Windows environment? Currently I use
>> WinSCP to initiate a connection to the EMR instance, then spawn a PuTTY
>> session.****
>>
>>  ****
>>
>> The changes to the hive config look identical to what I’ve changed in my
>> hive-default.xml. Does it make any difference which config file the
>> changesare applied to (hive-default.xml vs hive-site.xml)?****
>>
>>  ****
>>
>> Tony****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Ariel Marcus [mailto:[EMAIL PROTECTED]]
>> *Sent:* 18 January 2013 16:16****
>>
>>
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: HWI use on AWS/EMR****
>>
>>  ****
>>
>> Hey Tony,****
>>
>>  ****
>>
>> I would recommend using ssh port forwarding instead of making your hwi
>> publicly available on the internet. When you connect to the master node
>> using ssh you should use a command like the following:****
>>
>>  ****
>>
>> ssh -i /path/to/keyfile.pem -L 9999:localhost:9999 -l hadoop
>
Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB