Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Use of required keyword in protobuf messages


Copy link to this message
-
Re: Use of required keyword in protobuf messages
Good question, Lars.

I'm going off of the proposal (
http://wiki.apache.org/hadoop/Hbase/HBaseWireCompatibility
) and the slides at the meetup (
http://files.meetup.com/1350427/wire-compat%20%281%29.pptx).  If I missed
anything, I apologize.

Those list the minimum requirements to support the use cases and goals on
slides 4-6.  So, as written, there is no *guarantee* of support for rolling
upgrade between major versions.

Of course, there is nothing stopping us from adding that guarantee.  Or
from just guaranteeing it for specific versions (e.g. 0.96 to 0.98 in the
same manner as 0.92 to 0.94).  It should be much easier to achieve the
latter than in the past, once everything has been protobufed.  Perhaps we
should discuss in another thread?

So I'll withdraw my point that perhaps we could treat server-server and
client-server communication differently, but still submit the point that we
don't need to keep required fields around forever.

Greg

On Tue, Aug 7, 2012 at 5:23 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Huh?
>
> I thought we're allowing for a rolling upgrade between major versions.
> If server/server is only wire compatible across a minor version that won't
> be possible.
>
> (this might have been mentioned in the initial discussion we all had about
> this a while back and I might have just forgotten)
>
>
> -- Lars
>
>
> ----- Original Message -----
> From: Gregory Chanan <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc:
> Sent: Tuesday, August 7, 2012 1:33 PM
> Subject: Re: Use of required keyword in protobuf messages
>
> I agree with most of the comments here, particularly that at some point we
> should go through and review all the "required" fields.
>
> Another thing to keep in mind is that we have only guaranteed the following
> are compatible:
> - Client/Server across 1 major version
> - Server/Server different minor versions
>
> So we don't need to keep required fields for eons, necessarily.  And it may
> make sense to be looser about allowing "required" for server/server
> communication than for client/server.
>
> Greg
>
> On Tue, Aug 7, 2012 at 11:39 AM, Devaraj Das <[EMAIL PROTECTED]> wrote:
>
> > I too tend to make fields optional unless I am really convinced that the
> > field would live for eons. I agree with Google's philosophy in that
> regard.
> >
> > On Aug 6, 2012, at 3:39 PM, Chris Trezzo wrote:
> >
> > > Hi All,
> > >
> > > I was looking through the .proto files and noticed there are a lot of
> > > fields that are marked as required. I am by no means a protobuf expert,
> > but
> > > I was wondering what advantage do we actually get in making fields
> > required?
> > >
> > > I understand that if we don't use the required keyword we would have to
> > > implement custom application logic, but the flexibility we gain from
> > having
> > > all the fields optional seems to outweigh that work. In addition, we
> will
> > > already have to add logic to HBase to handle version compatibility, so
> it
> > > seems natural to implement the required logic as part of that layer.
> This
> > > would allow us to change or delete any message field and maintain wire
> > > compatibility.
> > >
> > > Quote from the protobuf language guide (
> > > https://developers.google.com/protocol-buffers/docs/proto):
> > >
> > > "*Required Is Forever* You should be very careful about marking fields
> as
> > > required. If at some point you wish to stop writing or sending a
> required
> > > field, it will be problematic to change the field to an optional field
> –
> > > old readers will consider messages without this field to be incomplete
> > and
> > > may reject or drop them unintentionally. You should consider writing
> > > application-specific custom validation routines for your buffers
> instead.
> > > Some engineers at Google have come to the conclusion that using
> > > requireddoes more harm than good; they prefer to use only
> > > optional and repeated. However, this view is not universal."
> > >
> > > Thoughts?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB