Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - How to use Apache Avro to encode the data in avro binary encoded form?


Copy link to this message
-
Re: How to use Apache Avro to encode the data in avro binary encoded form?
Raihan Jamal 2013-09-20, 21:44
Hello Mika/Doug,

I am thinking to write byteData as it is in one of my columns in Cassandra
Column Family so is this right? I have figured it out how can I store it in
Cassandra already..
I just wanted to confirm whether I can write byteData as it is or not?

*Raihan Jamal*
On Thu, Sep 19, 2013 at 10:45 AM, raihan26 <[EMAIL PROTECTED]> wrote:

> Thanks Mika. That worked fine
>
> GenericDatumWriter<GenericRecord> writer = new
> GenericDatumWriter<GenericRecord>(schema);
>         ByteArrayOutputStream os = new ByteArrayOutputStream();
>         Encoder e = EncoderFactory.get().binaryEncoder(os, null);
>         writer.write(record, e);
>         e.flush();
>         byte[] byteData = os.toByteArray();
>         os.close();
>
>
> That means, I can write byteData as the value for one of my columns.
> Correct? I will figure out whether we can store binary blobs or not in
> Cassandra. But just to make sure, I need to store byteData as it is as the
> value for one of my columns.. Right?
>
> And also how can I deserialize the above data after I retrieve from
> Cassandra for that particular column? A simple example will make me
> understand this as well. Thanks.
>
>
>
>
>
>
>
>
> *Raihan Jamal*
>
>
> On Thu, Sep 19, 2013 at 1:53 AM, Mika Ristimaki [via Apache Avro] <[hidden
> email] <http://user/SendEmail.jtp?type=node&node=4028224&i=0>> wrote:
>
>> Hi Raihan,
>>
>> You can try somthing like this, see below.
>>
>> On Sep 19, 2013, at 4:04 AM, Raihan Jamal <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028221&i=0>>
>> wrote:
>>
>> > I am trying to use Apache Avro to serialize one of our Attributes data
>> and I want to use binary encoding version for that not the JSON version. In
>> this case, I don't want to serialize to disk or some avro data files. I am
>> planning to write Avro binary encoded-data in Cassandra column family.
>> >
>> > Below is the code so far I have- I am not sure how to start encoding
>> into binary version.
>> >
>> >     Schema schema = new
>> Parser().parse((AvroExample.class.getResourceAsStream("/user.avsc")));
>> >
>> >     GenericRecord record = new GenericData.Record(schema);
>> >     record.put("vbs", "99,E,0");
>> >     record.put("eby", "0,E,,,,,,,15");
>> >     record.put("ste", "77;2,0;1,16;3");
>> >
>> >
>> >     // what I am supposed to do here to serialize into binary encoded
>> version of Avro?
>> >
>>         GenericDatumWriter<GenericRecord> writer = new
>> GenericDatumWriter<GenericRecord>(schema);
>>         ByteArrayOutputStream os = new ByteArrayOutputStream();
>>         Encoder e = EncoderFactory.get().binaryEncoder(os, null);
>>         writer.write(record, e);
>>         e.flush();
>>         byte[] byteData = os.toByteArray();
>>         os.close();
>> >
>> >    // then write to Cassandra here.
>> >
>>
>> I don't know anything about cassandra, but I'm pretty sure you can write
>> binary blobs there.
>>
>> -Mika
>>
>> >
>> > Can anyone provide a simple example basis on my code how to do that?
>> And then I can use that to write into Cassandra column family.
>> >
>> > Thanks.
>> >
>> >
>> >
>> >
>> > Raihan Jamal
>>
>>
>>
>> ------------------------------
>>  If you reply to this email, your message will be added to the
>> discussion below:
>>
>> http://apache-avro.679487.n3.nabble.com/How-to-use-Apache-Avro-to-encode-the-data-in-avro-binary-encoded-form-tp4028217p4028221.html
>>  To start a new topic under Avro - Users, email [hidden email]<http://user/SendEmail.jtp?type=node&node=4028224&i=1>
>> To unsubscribe from Apache Avro, click here.
>> NAML<http://apache-avro.679487.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ------------------------------
> View this message in context: Re: How to use Apache Avro to encode the