Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> HTTP transport?


Copy link to this message
-
Re: HTTP transport?

On Sep 28, 2009, at 3:42 PM, Doug Cutting wrote:

> Owen O'Malley wrote:
> > I've got concerns about this. Both tactical and strategic. The  
> tactical
> > problem is that I need to get security (both Kerberos and token)  
> in to
> > 0.22. I'd really like to get Avro RPC into 0.22. I'd like both to be
> > done roughly in 5 months. If you switch off of the current RPC  
> code base
> > to a completely new RPC code base, I don't see that happening.
>
> What transport do you expect to use with Avro?  If wire-
> compatibility is
> a goal, and that includes access from languages besides Java, then we
> must use a transport that's well-specified and Java-independent.  HTTP
> is both of these.  The existing Hadoop RPC protocol is not.
>
> We could adapting Hadoop's existing RPC transport to be well-specified
> and language independent.  This is perhaps not a huge task, but it  
> feels
> to me a bit like re-inventing much of what's already in HTTP clients  
> and
> servers these days: connection-pooling, async servers, etc.
>
Wrt  connection pooling/async servers: Can't we use the same libraries  
that Jetty and Tomcat use?
  Grizzly?

> grate Kerberos with
> Jetty than with a homegrown protocol and server?
>
>
> >   - very expensive on the wire encryption (ssl)
>
> If we don't use HTTP, will we be providing on-wire encryption?  If  
> not,
> this is moot.
>

Yes we are expecting to use encryption down the road.
>
> Finally, need to have secure HTTP-based access anyway, right?  If we  
> use
> HTTP as our RPC transport mightn't we reuse most of that effort?
>
> Doug
>