Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # user - Tunneling over SSH


+
stbill79@... 2013-09-05, 23:14
+
Christopher 2013-09-06, 02:36
+
Christopher 2013-09-06, 02:37
+
Eric Newton 2013-09-06, 03:33
Copy link to this message
-
Re: Tunneling over SSH
Christopher 2013-09-07, 01:17
That's a good point. You can run the thrift proxy on the remote node,
and use port forwarding over SSH to talk to it.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Thu, Sep 5, 2013 at 11:33 PM, Eric Newton <[EMAIL PROTECTED]> wrote:
> Speaking of Proxy... there's a Thrift Proxy that would accommodate a single
> port connection to do all client operations if hosted on the subnet.  Bonus:
> you can use any thrift-supported language.
>
> Without the proxy, however, the data model (inherent to the BigTable design)
> is that the client can reach every tablet server, as well as the pointer to
> the root tablet on the fault-tolerant register (zookeeper/chubby).  Without
> a network that supports this connectivity, you are fighting the
> architecture.
>
> -Eric
>
>
>
> On Thu, Sep 5, 2013 at 10:36 PM, Christopher <[EMAIL PROTECTED]> wrote:
>>
>> You're right that ZK is instructing the client to talk directly to
>> 192.168.182.22:9997 (tablet server). As Mike suggested, this could be
>> resolved if we stored hostnames rather than IPs, and you had hostnames
>> mapped to the external IP, and ports forwarded over SSH.
>>
>> A more robust solution would be to have a client-side configuration
>> setting that allowed you to specify a SOCKS proxy. The standard system
>> properties "socksProxyHost" and "socksProxyPort" may even work today,
>> if you set them up as system properties in your client code before you
>> open a thrift connection... I haven't tested this myself.
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>>
>> On Thu, Sep 5, 2013 at 7:14 PM,  <[EMAIL PROTECTED]> wrote:
>> > I'm trying to tunnel via SSH to a single Hadoop,Zoo, Accumulo
>> > stand-alone
>> > installation. The internal IP of the machine is on a local subnet behind
>> > a
>> > SSH-only firewall - 192.168.182.22.. I use static host names in all of
>> > the
>> > config files (Accumulo, Zoo, Hadoop) that resolve to 192.168.182.22 for
>> > all
>> > the servers. There is no problem connecting when I'm directly connected
>> > to
>> > the subnet inside the firewall.
>> >
>> > However, when I try to connect via the JAVA API from outside the
>> > firewall, I
>> > get an error: Failed to find an available server in the list of servers:
>> > [192.168.182.22:9997:9997 (120000)]. I've created a Windows Loopback
>> > interface that allows me to forward unlimited ports directly through the
>> > SSH
>> > tunnel to the internal network - there is no issue with connecting to
>> > Hadoop
>> > via Java or the web interface, and I can view the Accumuoo status page
>> > at
>> > 50095 by just setting my Windows box to resolve the hostname to the
>> > loopback
>> > local IP -> SSH -> 192.168.182.22:50095.
>> >
>> > I think the problem is that Zookeeper is telling my Java process to try
>> > and
>> > make a connection directly to 192.168.22.9997. If Zoo would use the
>> > hostname, there'd be no problem as it'd resolve to the loopback, and get
>> > tunneled along with everything else. But since it uses the actual IP,
>> > the
>> > Windows box won't route that back through the SSH tunnel as it considers
>> > it
>> > a local subnet outside of the firewall.
>> >
>> > Anyone experienced this issue and have a solution? I guess one solution
>> > might be to 'trick' Windows into forwarding the 192.168.x.y subnet back
>> > through the loopback (-> SSH), but I'm not seeing a good way to do that.
>> >
>> > Thanks
>
>
+
Mike Drob 2013-09-05, 23:33