Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Bypassing "handshake" in Responder


+
Pankaj Shroff 2013-02-25, 17:08
+
Doug Cutting 2013-02-25, 18:38
Copy link to this message
-
Re: Bypassing "handshake" in Responder
Doug

Perhaps you answered a portion of my conundrum in another thread (permalink
below) - but there is still the handshake and reuse of invocation logic
question. Let me also think about this a little bit.

Thanks in any case. Avro is a great tool in any case!

http://mail-archives.apache.org/mod_mbox/avro-user/201302.mbox/%3CCALEq1Z_rt8FasjSR%2B%2BOOgE3ogrAh0Y%2BtL3z47hznuiBAtfvWmw%40mail.gmail.com%3E
Pankaj

[EMAIL PROTECTED]

On Mon, Feb 25, 2013 at 1:38 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> This sounds like a different RPC wire format than Avro's.  Avro's
> Requestor and Responder implement Avro's RPC wire format.  Avro's
> Encode/Decoder and DatumReader/DatumWriter APIs should facilitate
> implementation of other RPC wire formats that include Avro data.
> Avro's Transceiver API may or may not be reusable, since it assumes
> Avro-style framing.  Parts of Requestor and Responder *might* be
> reusable and some refactoring of those classes *might* make such reuse
> easier, but there's not that much logic there that's not specific to
> Avro's wire format, so it might be just as easy to reimplement this
> layer for a different wire format.  It's hard for me to say without
> seeing a patch with a proposed refactoring.  Does that make sense?
>
> Doug
>
> On Mon, Feb 25, 2013 at 9:08 AM, Pankaj Shroff <[EMAIL PROTECTED]> wrote:
> > Hi
> >
> > We are using Avro for implementing an open source reference
> implementation
> > of the OpenRTB protocol.
> >
> > We have made a design goal to model the protocol using Avro protocol
> files
> > (.avpr) and generate types defined in the protocol schema using Avro .
> The
> > challenge is that the protocol does not necessarily require the use of
> Avro/
> > Binary wire encoding - or even the use of Avro/ RPC context. In fact many
> > counter parties have proprietary implementations supporting either
> Protobuf
> > or Json encoding.
> >
> > Now, there is a Json encoder/decoder in the Avro package but it seems
> that
> > the approach is a "schema-first" approach. The JsonEncoder assumes that
> the
> > encoding on the wire still follows the Avro Json encoding - which
> includes a
> > handshake followed by schema confirmation on both sides (client and
> server).
> >
> > For the protocol we are implementing - this presents 2 problems if Avro/
> > binary is not the chose encoding type for both sides - and if instead,
> lets
> > say, raw Json encoding is being used
> >
> > 1) the handshake is rather Avro specific - and we would like to
> completely
> > skip it if both sides have agreed on using raw json encoding - there may
> be
> > a separate info-exchange phase in the protocol that is protocol specific
> and
> > does not need to use Avro handshake. Is it possible to use Avro RPC
> without
> > the handshake?
> >
> > 2) we would like to use the data binding and schema resolution as
> > implemented by the SpecificResponder class in Avro - but extend it to use
> > raw JSON - not Avro JSON - encodings.
> >
> > 3) We would prefer not to have to override the "respond(List<buffers>)"
> > method of the base class Responder. This implementation always performs
> > handshake and always uses BinaryEncoder/Decoder which removes any
> > flexibility of using a different encoder /decoder in a derived class. We
> > would prefer if the Responder or some derived class saves the chosen
> > Decoder/ encoder as a protected property of the Responder object.
> Instead of
> > instantiating BinaryEncoder/ Decoder objects on the fly within the
> respond
> > method, it would be great if this was made more extensible and if the
> > Encoder/Decoder can be specified during construction.
> >
> > 4) For future flexibility it would be great to have the handshake
> > functionality available in sub-classes of Responder as an inherited
> method
> > (instead of private scope right now).
> >
> > I would welcome any suggestions/corrections.
> >
> > Pankaj
> >
> > --
> > Pankaj Shroff
> > twitter: @chompi
> > http://github.com/chompi

Pankaj Shroff
[EMAIL PROTECTED]
+
Pankaj Shroff 2013-02-27, 19:16
+
Doug Cutting 2013-02-27, 19:50
+
Pankaj Shroff 2013-02-27, 21:38
+
Pankaj Shroff 2013-03-12, 21:01
+
Doug Cutting 2013-03-13, 16:20
+
Pankaj Shroff 2013-02-25, 19:15