Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - Tunneling over SSH


Copy link to this message
-
Re: Tunneling over SSH
Eric Newton 2013-09-06, 03:33
Speaking of Proxy... there's a Thrift Proxy that would accommodate a single
port connection to do all client operations if hosted on the subnet.
 Bonus: you can use any thrift-supported language.

Without the proxy, however, the data model (inherent to the BigTable
design) is that the client can reach every tablet server, as well as the
pointer to the root tablet on the fault-tolerant register
(zookeeper/chubby).  Without a network that supports this connectivity, you
are fighting the architecture.

-Eric

On Thu, Sep 5, 2013 at 10:36 PM, Christopher <[EMAIL PROTECTED]> wrote:

> You're right that ZK is instructing the client to talk directly to
> 192.168.182.22:9997 (tablet server). As Mike suggested, this could be
> resolved if we stored hostnames rather than IPs, and you had hostnames
> mapped to the external IP, and ports forwarded over SSH.
>
> A more robust solution would be to have a client-side configuration
> setting that allowed you to specify a SOCKS proxy. The standard system
> properties "socksProxyHost" and "socksProxyPort" may even work today,
> if you set them up as system properties in your client code before you
> open a thrift connection... I haven't tested this myself.
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
>
> On Thu, Sep 5, 2013 at 7:14 PM,  <[EMAIL PROTECTED]> wrote:
> > I'm trying to tunnel via SSH to a single Hadoop,Zoo, Accumulo stand-alone
> > installation. The internal IP of the machine is on a local subnet behind
> a
> > SSH-only firewall - 192.168.182.22.. I use static host names in all of
> the
> > config files (Accumulo, Zoo, Hadoop) that resolve to 192.168.182.22 for
> all
> > the servers. There is no problem connecting when I'm directly connected
> to
> > the subnet inside the firewall.
> >
> > However, when I try to connect via the JAVA API from outside the
> firewall, I
> > get an error: Failed to find an available server in the list of servers:
> > [192.168.182.22:9997:9997 (120000)]. I've created a Windows Loopback
> > interface that allows me to forward unlimited ports directly through the
> SSH
> > tunnel to the internal network - there is no issue with connecting to
> Hadoop
> > via Java or the web interface, and I can view the Accumuoo status page at
> > 50095 by just setting my Windows box to resolve the hostname to the
> loopback
> > local IP -> SSH -> 192.168.182.22:50095.
> >
> > I think the problem is that Zookeeper is telling my Java process to try
> and
> > make a connection directly to 192.168.22.9997. If Zoo would use the
> > hostname, there'd be no problem as it'd resolve to the loopback, and get
> > tunneled along with everything else. But since it uses the actual IP, the
> > Windows box won't route that back through the SSH tunnel as it considers
> it
> > a local subnet outside of the firewall.
> >
> > Anyone experienced this issue and have a solution? I guess one solution
> > might be to 'trick' Windows into forwarding the 192.168.x.y subnet back
> > through the loopback (-> SSH), but I'm not seeing a good way to do that.
> >
> > Thanks
>