Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> HWI use on AWS/EMR


+
Tony Burton 2013-01-18, 15:54
+
Dean Wampler 2013-01-18, 16:06
+
Dean Wampler 2013-01-18, 16:09
+
Tony Burton 2013-01-18, 16:20
+
Ariel Marcus 2013-01-18, 16:15
+
Tony Burton 2013-01-18, 16:26
+
Ariel Marcus 2013-01-18, 16:31
+
Tony Burton 2013-01-18, 17:00
+
Ariel Marcus 2013-01-18, 17:05
+
Nitin Pawar 2013-01-18, 17:07
+
Tony Burton 2013-01-18, 17:17
+
Ariel Marcus 2013-01-18, 17:24
+
Tony Burton 2013-01-21, 11:39
+
Nitin Pawar 2013-01-21, 11:56
+
Tony Burton 2013-01-21, 13:13
+
Nitin Pawar 2013-01-21, 13:16
+
Tony Burton 2013-01-21, 16:37
Copy link to this message
-
Re: HWI use on AWS/EMR
Yes, this is straightforward to set up in Putty.  Ariel, thanks for
providing the details I was too lazy to mention ;)

One flaw with this approach is that only your machine will have access to
HWI. That is, the port is forwarded only to your machine. Making the ports
public on the master node means anyone on the interwebs can see it. You can
also set up security groups in AWS so that only specifically authorized
people can access the ports. That's a bit involved...

dean

On Fri, Jan 18, 2013 at 10:31 AM, Ariel Marcus <[EMAIL PROTECTED]>wrote:

> Hey Tony,
>
> Port forwarding is the same as what Dean referred to as ssh tunneling.
>
> Here is a website explaining how to set up port forwarding in putty (use
> port 9999):
> http://www.cs.uu.nl/technical/services/ssh/putty/puttyfw.html
>
> You can edit either hive-default or hive-site but the general practice is
> to store changes from the default configuration in hive-site.
>
> Best,
> Ariel
>
>
> ---------------------------------
> Ariel Marcus, Consultant
> www.openbi.com | [EMAIL PROTECTED]
> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
> Cell: 314-827-4356
>
>
> On Fri, Jan 18, 2013 at 11:26 AM, Tony Burton <[EMAIL PROTECTED]>wrote:
>
>> Hi Ariel,****
>>
>> ** **
>>
>> Thanks for the speedy reply. We’ll be accessing the HWI from Windows
>> rather Linux desktops, so can you help me out with understanding how to
>> carry out the port forwarding from a Windows environment? Currently I use
>> WinSCP to initiate a connection to the EMR instance, then spawn a PuTTY
>> session.****
>>
>> ** **
>>
>> The changes to the hive config look identical to what I’ve changed in my
>> hive-default.xml. Does it make any difference which config file the
>> changesare applied to (hive-default.xml vs hive-site.xml)?****
>>
>> ** **
>>
>> Tony****
>>
>> ** **
>>
>> ** **
>>
>> *From:* Ariel Marcus [mailto:[EMAIL PROTECTED]]
>> *Sent:* 18 January 2013 16:16
>>
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: HWI use on AWS/EMR****
>>
>> ** **
>>
>> Hey Tony,****
>>
>> ** **
>>
>> I would recommend using ssh port forwarding instead of making your hwi
>> publicly available on the internet. When you connect to the master node
>> using ssh you should use a command like the following:****
>>
>> ** **
>>
>> ssh -i /path/to/keyfile.pem -L 9999:localhost:9999 -l hadoop
>> MASTER_HOSTNAME****
>>
>> ** **
>>
>> After you have connected, add the following to your
>> hive/conf/hive-site.xml file:****
>>
>> ** **
>>
>> <property>****
>>
>>   <name>hive.hwi.war.file</name>****
>>
>>   <value>lib/hive-hwi-0.8.1.war</value>****
>>
>>   <description>This is the WAR file with the jsp content for Hive Web
>> Interface</description>****
>>
>> </property>****
>>
>> ** **
>>
>> Run this command to start up hwi:****
>>
>> ** **
>>
>> hive --service hwi****
>>
>> ** **
>>
>> And finally point your browser to:****
>>
>> ** **
>>
>> localhost:9999****
>>
>> ** **
>>
>> That worked for me.****
>>
>> ** **
>>
>> Best,****
>>
>> Ariel****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>>
>> ****
>>
>> ---------------------------------****
>>
>> Ariel Marcus, Consultant****
>>
>> www.openbi.com | [EMAIL PROTECTED]****
>>
>> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
>> Cell: 314-827-4356****
>>
>> ** **
>>
>> On Fri, Jan 18, 2013 at 11:09 AM, Dean Wampler <
>> [EMAIL PROTECTED]> wrote:****
>>
>> Oops, I overlooked that you have the public domain name in your message.
>> Can you surf to  http://ec2-54-247-61-206.eu-west-1.compute.amazonaws.com?  If not, does HWI use port 80? Is whatever port it uses blocked by EC2?
>> ****
>>
>> ** **
>>
>> If it's blocked you can use ssh to tunnel the port through.****
>>
>> ** **
>>
>> dean****
>>
>> ** **
>>
>> On Fri, Jan 18, 2013 at 10:06 AM, Dean Wampler <
>> [EMAIL PROTECTED]> wrote:****
>>
>> That's the internal hostname, not visible outside. Use the name like
>> ec2-NNN-NN-NN-NNN.compute-1.amazonaws.com. It's shown in the EMR console

*Dean Wampler, Ph.D.*
thinkbiganalytics.com
+1-312-339-1330