Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - HWI use on AWS/EMR


+
Tony Burton 2013-01-18, 15:54
+
Dean Wampler 2013-01-18, 16:06
+
Dean Wampler 2013-01-18, 16:09
+
Tony Burton 2013-01-18, 16:20
+
Ariel Marcus 2013-01-18, 16:15
+
Tony Burton 2013-01-18, 16:26
+
Ariel Marcus 2013-01-18, 16:31
+
Tony Burton 2013-01-18, 17:00
+
Ariel Marcus 2013-01-18, 17:05
Copy link to this message
-
Re: HWI use on AWS/EMR
Nitin Pawar 2013-01-18, 17:07
ok so much discussion around this but why dont you open access to port 9999
for your specific machine (home/office) and access it normally without
setting up port forwarding.

AWS does give you ip level access control in the security groups
On Fri, Jan 18, 2013 at 10:35 PM, Ariel Marcus <[EMAIL PROTECTED]>wrote:

> This should be localhost instead of proxy:
>
> - In Connection/SSH/Tunnels I’ve added 9999 in the Source Port box, *
> localhost*:9999 in the Destination box. Not sure if this is correct.
>
> Best,
> Aril
>
> ---------------------------------
> Ariel Marcus, Consultant
> www.openbi.com | [EMAIL PROTECTED]
> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
> Cell: 314-827-4356
>
>
> On Fri, Jan 18, 2013 at 12:00 PM, Tony Burton <[EMAIL PROTECTED]>wrote:
>
>> ** **
>>
>> Ok, thanks Ariel (and Dean in the next email!)****
>>
>> ** **
>>
>> I’ve set up PuTTY (v0.62) as follows:****
>>
>> - EMR instance hostname in the Session/Host Name box (port left at 22,
>> ok?)****
>>
>> - In Connection/SSH/Auth I’ve added my private key file, downloaded from
>> AWS ages for this account.****
>>
>> - In Connection/SSH/Tunnels I’ve added 9999 in the Source Port box,
>> proxy:9999 in the Destination box. Not sure if this is correct.****
>>
>> ** **
>>
>> Hit the Open button, I can connect. (And I can see my changes in
>> hive-default.xml, so I know I’m in the right place:)****
>>
>> Then pointing my browser at http://localhost:9999 I get the same timeout
>> again. Have I set up everything right?****
>>
>> ** **
>>
>> Also, following good advice I copied the config changes to hive-site.xml.
>> How do I stop hive so I can restart with ‘hive –service hwi’ so that it
>> picks up the new config?****
>>
>> ** **
>>
>> Tony****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> *From:* Ariel Marcus [mailto:[EMAIL PROTECTED]]
>> *Sent:* 18 January 2013 16:31
>>
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: HWI use on AWS/EMR****
>>
>> ** **
>>
>> Hey Tony,****
>>
>> ** **
>>
>> Port forwarding is the same as what Dean referred to as ssh tunneling.***
>> *
>>
>> ** **
>>
>> Here is a website explaining how to set up port forwarding in putty (use
>> port 9999):****
>>
>> http://www.cs.uu.nl/technical/services/ssh/putty/puttyfw.html****
>>
>> ** **
>>
>> You can edit either hive-default or hive-site but the general practice is
>> to store changes from the default configuration in hive-site.****
>>
>> ** **
>>
>> Best,****
>>
>> Ariel****
>>
>> ** **
>>
>>
>> ****
>>
>> ---------------------------------****
>>
>> Ariel Marcus, Consultant****
>>
>> www.openbi.com | [EMAIL PROTECTED]****
>>
>> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
>> Cell: 314-827-4356****
>>
>> ** **
>>
>> On Fri, Jan 18, 2013 at 11:26 AM, Tony Burton <[EMAIL PROTECTED]>
>> wrote:****
>>
>> Hi Ariel,****
>>
>>  ****
>>
>> Thanks for the speedy reply. We’ll be accessing the HWI from Windows
>> rather Linux desktops, so can you help me out with understanding how to
>> carry out the port forwarding from a Windows environment? Currently I use
>> WinSCP to initiate a connection to the EMR instance, then spawn a PuTTY
>> session.****
>>
>>  ****
>>
>> The changes to the hive config look identical to what I’ve changed in my
>> hive-default.xml. Does it make any difference which config file the
>> changesare applied to (hive-default.xml vs hive-site.xml)?****
>>
>>  ****
>>
>> Tony****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Ariel Marcus [mailto:[EMAIL PROTECTED]]
>> *Sent:* 18 January 2013 16:16****
>>
>>
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: HWI use on AWS/EMR****
>>
>>  ****
>>
>> Hey Tony,****
>>
>>  ****
>>
>> I would recommend using ssh port forwarding instead of making your hwi
>> publicly available on the internet. When you connect to the master node
>> using ssh you should use a command like the following:****
>>
>>  ****
>>
>> ssh -i /path/to/keyfile.pem -L 9999:localhost:9999 -l hadoop
>
Nitin Pawar
+
Tony Burton 2013-01-18, 17:17
+
Ariel Marcus 2013-01-18, 17:24
+
Tony Burton 2013-01-21, 11:39
+
Nitin Pawar 2013-01-21, 11:56
+
Tony Burton 2013-01-21, 13:13
+
Nitin Pawar 2013-01-21, 13:16
+
Tony Burton 2013-01-21, 16:37
+
Dean Wampler 2013-01-18, 16:38