Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Apache Avro Encoding types?


Copy link to this message
-
Re: Apache Avro Encoding types?
Hi Raihan,

One strategy may be to store a cache of schemata elsewhere and then write just enough information (version number, md5, or similar) to look up the right schema version adjacent to your avro/binary blob column.  This would impose little overhead and still allow you to take advantage of Avro's built-in schema resolution to read old serialized data.
--
Connor

> On Sep 18, 2013, at 20:03, raihan26 <[EMAIL PROTECTED]> wrote:
>
> Thanks a lot Doug. So that means, If I need to store avro binary encoded data in Cassandra column family, then the actual schema won't get stored with that? Right? I need to have the schema through some external means whenever I am reading that binary encoded data. Right?
>
> Raihan Jamal
>
>
>> On Wed, Sep 18, 2013 at 2:28 PM, Doug Cutting [via Apache Avro] <[hidden email]> wrote:
>> On Wed, Sep 18, 2013 at 1:53 PM, Raihan Jamal <[hidden email]> wrote:
>> > Suppose if I am going with Binary encoding route instead of JSON then does
>> > the schema also gets stored in that binary encoding format always?
>>
>> The schema is stored in Avro data files.  Avro RPC also manages
>> schemas for you.  But if you store Avro binary-encoded data in some
>> other container then you may need to keep track of the schema
>> yourself.
>>
>> Doug
>>
>>
>> If you reply to this email, your message will be added to the discussion below:
>> http://apache-avro.679487.n3.nabble.com/Apache-Avro-Encoding-types-tp4028194p4028215.html
>> To start a new topic under Avro - Users, email [hidden email]
>> To unsubscribe from Apache Avro, click here.
>> NAML
>
>
> View this message in context: Re: Apache Avro Encoding types?
> Sent from the Avro - Users mailing list archive at Nabble.com.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB