Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # dev >> A couple questions on the RPC spec.


+
Jeremy Custenborder 2010-06-07, 21:25
+
Doug Cutting 2010-06-07, 22:03
+
Jeremy Custenborder 2010-06-08, 01:23
Copy link to this message
-
Re: A couple questions on the RPC spec.
Hey Jeremy,

The metadata is part of the call request record, which in Avro is a ["null",
{"type": "map", "values": "bytes"}] followed by a "string" followed by a
nameless record of the message parameters (and yes, we should specify this
protocol in Avro; see
https://issues.apache.org/jira/browse/AVRO-341?focusedCommentId=12831204&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12831204
).

Thus I think the latter option looks correct; Doug can tell you
definitively.

Later,
Jeff

On Mon, Jun 7, 2010 at 6:23 PM, Jeremy Custenborder <[EMAIL PROTECTED]
> wrote:

> Thanks Doug,
>
> That affirms the direction that I was heading. Should I assume that
> metadata
> is unique to each message? Here is what I have done so far for the code
> generator portion. Should I worry about generating code that looks like
> example 1 where I pass along a dictionary for metadata?  Or should I look
> at
> something like example 2 where I don't pass along the metadata. I could
> also
> do a combination of both.
>
> Example 1
> [System.CodeDom.Compiler.GeneratedCodeAttribute("Avro.CodeGen.AvroGen",
> "1.0")]
>    public interface Simple
>    {
>        /// <summary>
>        /// Send a greeting
>        /// </summary>
>        string hello(System.Collections.Generic.IDictionary<string, byte[]>
> metadata, string greeting);
>        /// <summary>
>        /// Pretend you're in a cave!
>        /// </summary>
>        TestRecord echo(System.Collections.Generic.IDictionary<string,
> byte[]> metadata, TestRecord record);
>        int add(System.Collections.Generic.IDictionary<string, byte[]>
> metadata, int arg1, int arg2);
>        byte[] echoBytes(System.Collections.Generic.IDictionary<string,
> byte[]> metadata, byte[] data);
>        /// <summary>
>        /// Always throws an error.
>        /// </summary>
>        void error(System.Collections.Generic.IDictionary<string, byte[]>
> metadata);
>    }
>
> Example 2
>    [System.CodeDom.Compiler.GeneratedCodeAttribute("Avro.CodeGen.AvroGen",
> "1.0")]
>    public interface Simple
>    {
>        /// <summary>
>        /// Send a greeting
>        /// </summary>
>        string hello(string greeting);
>        /// <summary>
>        /// Pretend you're in a cave!
>        /// </summary>
>        TestRecord echo(TestRecord record);
>        int add(int arg1, int arg2);
>        byte[] echoBytes(byte[] data);
>        /// <summary>
>        /// Always throws an error.
>        /// </summary>
>        void error();
>     }
>
> On Mon, Jun 7, 2010 at 5:03 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>
> > On 06/07/2010 02:25 PM, Jeremy Custenborder wrote:
> >
> >> This will be my last question for at least the next day or two. :) I
> just
> >> want to double check my interpretation of the message framing.
> >>
> >
> > Framing is mostly orthogonal to payload structure.  It just says that the
> > bytes that make up a payload are broken into length-prefixed frames.
>  Those
> > frames may or may not align with the payload's structure.  When they do
> line
> > up, optimizations are possible, so implementations should try to line
> them
> > up when sending largish binary objects.
> >
> > Request payloads are of the form <metadata><messageName><parameters> and
> > response payloads are of the form <metadata><boolean><errorOrResponse>.
> >  These may be delivered in one or more frames.  A good framing might be
> to
> > use a single frame unless a parameter has a bytes value that's larger
> than
> > 4k, in which case that buffer might be transmitted as a distinct frame,
> so
> > that it can be potentially processed in zerocopy style.  I.e., if it
> > contains file data, then the sendfile system call could be used to copy
> data
> > from file to socket.
> >
> > Does that help?
> >
> > Doug
> >
>
+
Doug Cutting 2010-06-08, 16:37
+
Jeremy Custenborder 2010-06-08, 16:50
+
Doug Cutting 2010-06-08, 17:35
+
Jeremy Custenborder 2010-06-08, 18:38
+
Jeff Hammerbacher 2010-06-08, 18:53
+
Jeremy Custenborder 2010-06-08, 19:21