Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Bypassing "handshake" in Responder


+
Pankaj Shroff 2013-02-25, 17:08
+
Doug Cutting 2013-02-25, 18:38
Copy link to this message
-
Re: Bypassing "handshake" in Responder
Doug

Perhaps you answered a portion of my conundrum in another thread (permalink
below) - but there is still the handshake and reuse of invocation logic
question. Let me also think about this a little bit.

Thanks in any case. Avro is a great tool in any case!

http://mail-archives.apache.org/mod_mbox/avro-user/201302.mbox/%3CCALEq1Z_rt8FasjSR%2B%2BOOgE3ogrAh0Y%2BtL3z47hznuiBAtfvWmw%40mail.gmail.com%3E
Pankaj

[EMAIL PROTECTED]

On Mon, Feb 25, 2013 at 1:38 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> This sounds like a different RPC wire format than Avro's.  Avro's
> Requestor and Responder implement Avro's RPC wire format.  Avro's
> Encode/Decoder and DatumReader/DatumWriter APIs should facilitate
> implementation of other RPC wire formats that include Avro data.
> Avro's Transceiver API may or may not be reusable, since it assumes
> Avro-style framing.  Parts of Requestor and Responder *might* be
> reusable and some refactoring of those classes *might* make such reuse
> easier, but there's not that much logic there that's not specific to
> Avro's wire format, so it might be just as easy to reimplement this
> layer for a different wire format.  It's hard for me to say without
> seeing a patch with a proposed refactoring.  Does that make sense?
>
> Doug
>
> On Mon, Feb 25, 2013 at 9:08 AM, Pankaj Shroff <[EMAIL PROTECTED]> wrote:
> > Hi
> >
> > We are using Avro for implementing an open source reference
> implementation
> > of the OpenRTB protocol.
> >
> > We have made a design goal to model the protocol using Avro protocol
> files
> > (.avpr) and generate types defined in the protocol schema using Avro .
> The
> > challenge is that the protocol does not necessarily require the use of
> Avro/
> > Binary wire encoding - or even the use of Avro/ RPC context. In fact many
> > counter parties have proprietary implementations supporting either
> Protobuf
> > or Json encoding.
> >
> > Now, there is a Json encoder/decoder in the Avro package but it seems
> that
> > the approach is a "schema-first" approach. The JsonEncoder assumes that
> the
> > encoding on the wire still follows the Avro Json encoding - which
> includes a
> > handshake followed by schema confirmation on both sides (client and
> server).
> >
> > For the protocol we are implementing - this presents 2 problems if Avro/
> > binary is not the chose encoding type for both sides - and if instead,
> lets
> > say, raw Json encoding is being used
> >
> > 1) the handshake is rather Avro specific - and we would like to
> completely
> > skip it if both sides have agreed on using raw json encoding - there may
> be
> > a separate info-exchange phase in the protocol that is protocol specific
> and
> > does not need to use Avro handshake. Is it possible to use Avro RPC
> without
> > the handshake?
> >
> > 2) we would like to use the data binding and schema resolution as
> > implemented by the SpecificResponder class in Avro - but extend it to use
> > raw JSON - not Avro JSON - encodings.
> >
> > 3) We would prefer not to have to override the "respond(List<buffers>)"
> > method of the base class Responder. This implementation always performs
> > handshake and always uses BinaryEncoder/Decoder which removes any
> > flexibility of using a different encoder /decoder in a derived class. We
> > would prefer if the Responder or some derived class saves the chosen
> > Decoder/ encoder as a protected property of the Responder object.
> Instead of
> > instantiating BinaryEncoder/ Decoder objects on the fly within the
> respond
> > method, it would be great if this was made more extensible and if the
> > Encoder/Decoder can be specified during construction.
> >
> > 4) For future flexibility it would be great to have the handshake
> > functionality available in sub-classes of Responder as an inherited
> method
> > (instead of private scope right now).
> >
> > I would welcome any suggestions/corrections.
> >
> > Pankaj
> >
> > --
> > Pankaj Shroff
> > twitter: @chompi
> > http://github.com/chompi

Pankaj Shroff
[EMAIL PROTECTED]
+
Pankaj Shroff 2013-02-27, 19:16
+
Doug Cutting 2013-02-27, 19:50
+
Pankaj Shroff 2013-02-27, 21:38
+
Pankaj Shroff 2013-03-12, 21:01
+
Doug Cutting 2013-03-13, 16:20
+
Pankaj Shroff 2013-02-25, 19:15
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB