Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # dev >> A couple questions on the RPC spec.


+
Jeremy Custenborder 2010-06-07, 21:25
+
Doug Cutting 2010-06-07, 22:03
+
Jeremy Custenborder 2010-06-08, 01:23
Copy link to this message
-
Re: A couple questions on the RPC spec.
Hey Jeremy,

The metadata is part of the call request record, which in Avro is a ["null",
{"type": "map", "values": "bytes"}] followed by a "string" followed by a
nameless record of the message parameters (and yes, we should specify this
protocol in Avro; see
https://issues.apache.org/jira/browse/AVRO-341?focusedCommentId=12831204&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12831204
).

Thus I think the latter option looks correct; Doug can tell you
definitively.

Later,
Jeff

On Mon, Jun 7, 2010 at 6:23 PM, Jeremy Custenborder <[EMAIL PROTECTED]
> wrote:

> Thanks Doug,
>
> That affirms the direction that I was heading. Should I assume that
> metadata
> is unique to each message? Here is what I have done so far for the code
> generator portion. Should I worry about generating code that looks like
> example 1 where I pass along a dictionary for metadata?  Or should I look
> at
> something like example 2 where I don't pass along the metadata. I could
> also
> do a combination of both.
>
> Example 1
> [System.CodeDom.Compiler.GeneratedCodeAttribute("Avro.CodeGen.AvroGen",
> "1.0")]
>    public interface Simple
>    {
>        /// <summary>
>        /// Send a greeting
>        /// </summary>
>        string hello(System.Collections.Generic.IDictionary<string, byte[]>
> metadata, string greeting);
>        /// <summary>
>        /// Pretend you're in a cave!
>        /// </summary>
>        TestRecord echo(System.Collections.Generic.IDictionary<string,
> byte[]> metadata, TestRecord record);
>        int add(System.Collections.Generic.IDictionary<string, byte[]>
> metadata, int arg1, int arg2);
>        byte[] echoBytes(System.Collections.Generic.IDictionary<string,
> byte[]> metadata, byte[] data);
>        /// <summary>
>        /// Always throws an error.
>        /// </summary>
>        void error(System.Collections.Generic.IDictionary<string, byte[]>
> metadata);
>    }
>
> Example 2
>    [System.CodeDom.Compiler.GeneratedCodeAttribute("Avro.CodeGen.AvroGen",
> "1.0")]
>    public interface Simple
>    {
>        /// <summary>
>        /// Send a greeting
>        /// </summary>
>        string hello(string greeting);
>        /// <summary>
>        /// Pretend you're in a cave!
>        /// </summary>
>        TestRecord echo(TestRecord record);
>        int add(int arg1, int arg2);
>        byte[] echoBytes(byte[] data);
>        /// <summary>
>        /// Always throws an error.
>        /// </summary>
>        void error();
>     }
>
> On Mon, Jun 7, 2010 at 5:03 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>
> > On 06/07/2010 02:25 PM, Jeremy Custenborder wrote:
> >
> >> This will be my last question for at least the next day or two. :) I
> just
> >> want to double check my interpretation of the message framing.
> >>
> >
> > Framing is mostly orthogonal to payload structure.  It just says that the
> > bytes that make up a payload are broken into length-prefixed frames.
>  Those
> > frames may or may not align with the payload's structure.  When they do
> line
> > up, optimizations are possible, so implementations should try to line
> them
> > up when sending largish binary objects.
> >
> > Request payloads are of the form <metadata><messageName><parameters> and
> > response payloads are of the form <metadata><boolean><errorOrResponse>.
> >  These may be delivered in one or more frames.  A good framing might be
> to
> > use a single frame unless a parameter has a bytes value that's larger
> than
> > 4k, in which case that buffer might be transmitted as a distinct frame,
> so
> > that it can be potentially processed in zerocopy style.  I.e., if it
> > contains file data, then the sendfile system call could be used to copy
> data
> > from file to socket.
> >
> > Does that help?
> >
> > Doug
> >
>
+
Doug Cutting 2010-06-08, 16:37
+
Jeremy Custenborder 2010-06-08, 16:50
+
Doug Cutting 2010-06-08, 17:35
+
Jeremy Custenborder 2010-06-08, 18:38
+
Jeff Hammerbacher 2010-06-08, 18:53
+
Jeremy Custenborder 2010-06-08, 19:21
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB