Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - New version of Avro tap for Cascading


Copy link to this message
-
New version of Avro tap for Cascading
Ken Krugler 2012-10-29, 19:41
Hi all,

Cross-posting Chris Sever's recent post to the Cascading list…

-- Ken

> Hi everyone,
>
> We put out a new shiny version of cascading.avro. It's a mash of the previous cascading.avro and cascading-avro projects with some new features.
> https://github.com/bixolabs/cascading.avro
>
> Highlights:
> - Read avros without specifying a schema (can't do this with write yet but it's in the works)
> - Construct the AvroScheme with either an avro schema (new style) or by specifying a list of fields and types (old style).
> - Supports all avro types (except union of two non-null types), including nested records as cascading tuples.
> - User can specify avro maps and arrays either with a cascading tuple or a java list/map.
> - New option to tell cascading avro to not pack or unpack your avros when reading/writing. Use this if you want the actual Avro Record in hand or you provide a single output field containing an Avro record (similar to how WritableSequenceFile is handled).
>
> 2.1 is up on conjars now:
> http://conjars.org/cascading.avro/avro-scheme
>
> Here is a quick modification of the Cascading for the Impatient lessons 1 and 2 using Avro:
> https://gist.github.com/3975481
>
> Please give it a shot and let us know what breaks.
--------------------------------------------
http://about.me/kkrugler
+1 530-210-6378