Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro Container file and JsonEncoding.


Copy link to this message
-
Re: Avro Container file and JsonEncoding.
Scott Carey 2012-02-08, 17:57


On 2/8/12 7:14 AM, "karthik ramachandran" <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I'm trying to figure out if its possible to create an Avro container file with
> JsonEnconding.  It doesn't appear to be: org.apache.avro.file.DataFileWriter
> seems to use a binary encoder by default.

One thing to note is that if you write it to an Avro container file in
binary it will be significantly smaller.  You can extract the contents as
JSON using either the C command line tools or the Java 'tojson' tool.  If
the reason you want it in JSON is for human readability, this is all you
need.  

For example, I often do the following:

java ­jar avro-tools.jar tojson my_avro_file.avro | grep  Š.

or pipe it to other tools to view or interpret as JSON.

>
> Is there another FileWriter class that I should be using?

See Doug's comments.  It doesn't make sense to store JSON in an Avro Data
File because it is delimited with binary markers and contains binary
metadata.
>
>
> Karthik
>
> --
> Karthik Ramachandran
>