|
|
-
New version of Avro tap for CascadingKen Krugler 2012-10-29, 19:41
Hi all,
Cross-posting Chris Sever's recent post to the Cascading list… -- Ken > Hi everyone, > > We put out a new shiny version of cascading.avro. It's a mash of the previous cascading.avro and cascading-avro projects with some new features. > https://github.com/bixolabs/cascading.avro > > Highlights: > - Read avros without specifying a schema (can't do this with write yet but it's in the works) > - Construct the AvroScheme with either an avro schema (new style) or by specifying a list of fields and types (old style). > - Supports all avro types (except union of two non-null types), including nested records as cascading tuples. > - User can specify avro maps and arrays either with a cascading tuple or a java list/map. > - New option to tell cascading avro to not pack or unpack your avros when reading/writing. Use this if you want the actual Avro Record in hand or you provide a single output field containing an Avro record (similar to how WritableSequenceFile is handled). > > 2.1 is up on conjars now: > http://conjars.org/cascading.avro/avro-scheme > > Here is a quick modification of the Cascading for the Impatient lessons 1 and 2 using Avro: > https://gist.github.com/3975481 > > Please give it a shot and let us know what breaks. -------------------------------------------- http://about.me/kkrugler +1 530-210-6378 |