Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Apache Avro Encoding types?


Copy link to this message
-
Re: Apache Avro Encoding types?
Thanks Connor. That makes sense... That whats we are planning to do. We
will have Cassandra column which will have composite columns. And those
composite column will have-

Avro-Binary-Encoded_value   Avro-Schema-Name-For-This particular-column
Last-Modified-Date

So when we will retrieve the data for a particular USER-ID for the above
column, we will know its Schema-Name, then we can pull the actual schema
basis on this schema-name, and then we can deserialize the
Avro-Binary-Encoded_value using that schema.
*Raihan Jamal*
On Wed, Sep 18, 2013 at 7:15 PM, Connor Doyle <[EMAIL PROTECTED]> wrote:

> Hi Raihan,
>
> One strategy may be to store a cache of schemata elsewhere and then write
> just enough information (version number, md5, or similar) to look up the
> right schema version adjacent to your avro/binary blob column.  This would
> impose little overhead and still allow you to take advantage of Avro's
> built-in schema resolution to read old serialized data.
> --
> Connor
>
> On Sep 18, 2013, at 20:03, raihan26 <[EMAIL PROTECTED]> wrote:
>
> Thanks a lot Doug. So that means, If I need to store avro binary encoded
> data in Cassandra column family, then the actual schema won't get stored
> with that? Right? I need to have the schema through some external means
> whenever I am reading that binary encoded data. Right?
>
> *Raihan Jamal*
>
>
> On Wed, Sep 18, 2013 at 2:28 PM, Doug Cutting [via Apache Avro] <[hidden
> email] <http://user/SendEmail.jtp?type=node&node=4028216&i=0>> wrote:
>
>> On Wed, Sep 18, 2013 at 1:53 PM, Raihan Jamal <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028215&i=0>>
>> wrote:
>> > Suppose if I am going with Binary encoding route instead of JSON then
>> does
>> > the schema also gets stored in that binary encoding format always?
>>
>> The schema is stored in Avro data files.  Avro RPC also manages
>> schemas for you.  But if you store Avro binary-encoded data in some
>> other container then you may need to keep track of the schema
>> yourself.
>>
>> Doug
>>
>>
>> ------------------------------
>>  If you reply to this email, your message will be added to the
>> discussion below:
>>
>> http://apache-avro.679487.n3.nabble.com/Apache-Avro-Encoding-types-tp4028194p4028215.html
>>  To start a new topic under Avro - Users, email [hidden email]<http://user/SendEmail.jtp?type=node&node=4028216&i=1>
>> To unsubscribe from Apache Avro, click here.
>> NAML<http://apache-avro.679487.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ------------------------------
> View this message in context: Re: Apache Avro Encoding types?<http://apache-avro.679487.n3.nabble.com/Apache-Avro-Encoding-types-tp4028194p4028216.html>
> Sent from the Avro - Users mailing list archive<http://apache-avro.679487.n3.nabble.com/Avro-Users-f679479.html>at
> Nabble.com.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB