Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Bypassing "handshake" in Responder


Copy link to this message
-
Bypassing "handshake" in Responder
Pankaj Shroff 2013-02-25, 17:08
Hi

We are using Avro for implementing an open source reference implementation
of the OpenRTB protocol.

We have made a design goal to model the protocol using Avro protocol files
(.avpr) and generate types defined in the protocol schema using Avro . The
challenge is that the protocol does not necessarily require the use of
Avro/ Binary wire encoding - or even the use of Avro/ RPC context. In fact
many counter parties have proprietary implementations supporting either
Protobuf or Json encoding.

Now, there is a Json encoder/decoder in the Avro package but it seems that
the approach is a "schema-first" approach. The JsonEncoder assumes that the
encoding on the wire still follows the Avro Json encoding - which includes
a handshake followed by schema confirmation on both sides (client and
server).

For the protocol we are implementing - this presents 2 problems if Avro/
binary is not the chose encoding type for both sides - and if instead, lets
say, raw Json encoding is being used

1) the handshake is rather Avro specific - and we would like to completely
skip it if both sides have agreed on using raw json encoding - there may be
a separate info-exchange phase in the protocol that is protocol specific
and does not need to use Avro handshake. Is it possible to use Avro RPC
without the handshake?

2) we would like to use the data binding and schema resolution as
implemented by the SpecificResponder class in Avro - but extend it to use
raw JSON - not Avro JSON - encodings.

3) We would prefer not to have to override the "respond(List<buffers>)"
method of the base class Responder. This implementation always performs
handshake and always uses BinaryEncoder/Decoder which removes any
flexibility of using a different encoder /decoder in a derived class. We
would prefer if the Responder or some derived class saves the chosen
Decoder/ encoder as a protected property of the Responder object. Instead
of instantiating BinaryEncoder/ Decoder objects on the fly within the
respond method, it would be great if this was made more extensible and if
the Encoder/Decoder can be specified during construction.

4) For future flexibility it would be great to have the handshake
functionality available in sub-classes of Responder as an inherited method
(instead of private scope right now).

I would welcome any suggestions/corrections.

Pankaj

--
Pankaj Shroff
twitter: @chompi
http://github.com/chompi
http://github.com/openrtb/openrtb2x