Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Bypassing "handshake" in Responder


Copy link to this message
-
Re: Bypassing "handshake" in Responder
This sounds like a different RPC wire format than Avro's.  Avro's
Requestor and Responder implement Avro's RPC wire format.  Avro's
Encode/Decoder and DatumReader/DatumWriter APIs should facilitate
implementation of other RPC wire formats that include Avro data.
Avro's Transceiver API may or may not be reusable, since it assumes
Avro-style framing.  Parts of Requestor and Responder *might* be
reusable and some refactoring of those classes *might* make such reuse
easier, but there's not that much logic there that's not specific to
Avro's wire format, so it might be just as easy to reimplement this
layer for a different wire format.  It's hard for me to say without
seeing a patch with a proposed refactoring.  Does that make sense?

Doug

On Mon, Feb 25, 2013 at 9:08 AM, Pankaj Shroff <[EMAIL PROTECTED]> wrote:
> Hi
>
> We are using Avro for implementing an open source reference implementation
> of the OpenRTB protocol.
>
> We have made a design goal to model the protocol using Avro protocol files
> (.avpr) and generate types defined in the protocol schema using Avro . The
> challenge is that the protocol does not necessarily require the use of Avro/
> Binary wire encoding - or even the use of Avro/ RPC context. In fact many
> counter parties have proprietary implementations supporting either Protobuf
> or Json encoding.
>
> Now, there is a Json encoder/decoder in the Avro package but it seems that
> the approach is a "schema-first" approach. The JsonEncoder assumes that the
> encoding on the wire still follows the Avro Json encoding - which includes a
> handshake followed by schema confirmation on both sides (client and server).
>
> For the protocol we are implementing - this presents 2 problems if Avro/
> binary is not the chose encoding type for both sides - and if instead, lets
> say, raw Json encoding is being used
>
> 1) the handshake is rather Avro specific - and we would like to completely
> skip it if both sides have agreed on using raw json encoding - there may be
> a separate info-exchange phase in the protocol that is protocol specific and
> does not need to use Avro handshake. Is it possible to use Avro RPC without
> the handshake?
>
> 2) we would like to use the data binding and schema resolution as
> implemented by the SpecificResponder class in Avro - but extend it to use
> raw JSON - not Avro JSON - encodings.
>
> 3) We would prefer not to have to override the "respond(List<buffers>)"
> method of the base class Responder. This implementation always performs
> handshake and always uses BinaryEncoder/Decoder which removes any
> flexibility of using a different encoder /decoder in a derived class. We
> would prefer if the Responder or some derived class saves the chosen
> Decoder/ encoder as a protected property of the Responder object. Instead of
> instantiating BinaryEncoder/ Decoder objects on the fly within the respond
> method, it would be great if this was made more extensible and if the
> Encoder/Decoder can be specified during construction.
>
> 4) For future flexibility it would be great to have the handshake
> functionality available in sub-classes of Responder as an inherited method
> (instead of private scope right now).
>
> I would welcome any suggestions/corrections.
>
> Pankaj
>
> --
> Pankaj Shroff
> twitter: @chompi
> http://github.com/chompi
> http://github.com/openrtb/openrtb2x
>