Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Bypassing "handshake" in Responder


Copy link to this message
-
Re: Bypassing "handshake" in Responder
This looks reasonable to me.

Cheers,

Doug

On Tue, Mar 12, 2013 at 2:01 PM, Pankaj Shroff <[EMAIL PROTECTED]> wrote:
> Quick update on this issue. I think I finally figured out a way to bypass
> Avro style "handshake" when employing "custom" format or content type
> implementation but still trying to reuse or benefit from the
> Serialize/Deserialize capabilities of Avro. Perhaps the following usage is
> the "intended" REUSE use case in Avro. However, what I have below completely
> bypasses Avro RPC classes in the avro-ipc package. Let me know if this still
> mandates a patch proposal on Jira. Basically - I implemented my own
> "Responder-Server" combination without relying on reflection based method
> invocation of "classic" Avro. Link to source code below:
>
> https://github.com/openrtb/openrtb2x/blob/2.0/demand-side/dsp-core/src/main/java/org/openrtb/dsp/core/DemandSideServer.java
>
> Pankaj
>
>
>
> On Wed, Feb 27, 2013 at 4:38 PM, Pankaj Shroff <[EMAIL PROTECTED]> wrote:
>>
>> Doug - thanks again.
>>
>> I agree with you. I have been looking into it for the past few days and it
>> seems like this will require quite a bit of refactoring. I will try to
>> follow up on Jira with more specific details.
>>
>> Thanks
>> Pankaj
>>
>>
>>
>> On Wed, Feb 27, 2013 at 2:50 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>>>
>>> Pankaj,
>>>
>>> Avro RPC is currently specified to always uses the binary encoding
>>> (like Avro data files).   This might reasonably be extended to JSON.
>>> First we'd need to specify the new wire format.  Probably Avro's
>>> framing would not make sense for JSON-encoded RPC over HTTP.  Then
>>> we'd need to figure what of the existing Java implementation might be
>>> reused or adapted.  At a glance, it doesn't look to me like a few
>>> one-line changes would suffice, adding methods where things are
>>> hardwired, that rather more substantial changes would be required, but
>>> I might be missing something.  If you're interested in pursuing this
>>> then please file an issue in Jira where you can propose changes to the
>>> specification and implementation.
>>>
>>> Cheers,
>>>
>>> Doug
>>>
>>> On Wed, Feb 27, 2013 at 11:16 AM, Pankaj Shroff <[EMAIL PROTECTED]>
>>> wrote:
>>> > I guess my question is more basic - given that this is somewhat
>>> > specific to
>>> > my own use case:
>>> >
>>> > How does one use other forms of Encoder/Decoder implementations that
>>> > are
>>> > available in the Avro library along with the Avro-Ipc SDK.
>>> >
>>> > As of 1.7.3, I see that the only Encoding/Decoding that Avro-ipc
>>> > supports is
>>> > the BinaryEncoding
>>> >
>>> > Pankaj
>>> >
>>> >
>>> >
>>> > On Mon, Feb 25, 2013 at 2:25 PM, Pankaj Shroff <[EMAIL PROTECTED]>
>>> > wrote:
>>> >>
>>> >> Doug
>>> >>
>>> >> Perhaps you answered a portion of my conundrum in another thread
>>> >> (permalink below) - but there is still the handshake and reuse of
>>> >> invocation
>>> >> logic question. Let me also think about this a little bit.
>>> >>
>>> >> Thanks in any case. Avro is a great tool in any case!
>>> >>
>>> >>
>>> >>
>>> >> http://mail-archives.apache.org/mod_mbox/avro-user/201302.mbox/%3CCALEq1Z_rt8FasjSR%2B%2BOOgE3ogrAh0Y%2BtL3z47hznuiBAtfvWmw%40mail.gmail.com%3E
>>> >>
>>> >>
>>> >> Pankaj
>>> >>
>>> >> [EMAIL PROTECTED]
>>> >>
>>> >>
>>> >>
>>> >> On Mon, Feb 25, 2013 at 1:38 PM, Doug Cutting <[EMAIL PROTECTED]>
>>> >> wrote:
>>> >>>
>>> >>> This sounds like a different RPC wire format than Avro's.  Avro's
>>> >>> Requestor and Responder implement Avro's RPC wire format.  Avro's
>>> >>> Encode/Decoder and DatumReader/DatumWriter APIs should facilitate
>>> >>> implementation of other RPC wire formats that include Avro data.
>>> >>> Avro's Transceiver API may or may not be reusable, since it assumes
>>> >>> Avro-style framing.  Parts of Requestor and Responder *might* be
>>> >>> reusable and some refactoring of those classes *might* make such
>>> >>> reuse
>>> >>> easier, but there's not that much logic there that's not specific to