Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> New version of Avro tap for Cascading


Copy link to this message
-
New version of Avro tap for Cascading
Hi all,

Cross-posting Chris Sever's recent post to the Cascading list…

-- Ken

> Hi everyone,
>
> We put out a new shiny version of cascading.avro. It's a mash of the previous cascading.avro and cascading-avro projects with some new features.
> https://github.com/bixolabs/cascading.avro
>
> Highlights:
> - Read avros without specifying a schema (can't do this with write yet but it's in the works)
> - Construct the AvroScheme with either an avro schema (new style) or by specifying a list of fields and types (old style).
> - Supports all avro types (except union of two non-null types), including nested records as cascading tuples.
> - User can specify avro maps and arrays either with a cascading tuple or a java list/map.
> - New option to tell cascading avro to not pack or unpack your avros when reading/writing. Use this if you want the actual Avro Record in hand or you provide a single output field containing an Avro record (similar to how WritableSequenceFile is handled).
>
> 2.1 is up on conjars now:
> http://conjars.org/cascading.avro/avro-scheme
>
> Here is a quick modification of the Cascading for the Impatient lessons 1 and 2 using Avro:
> https://gist.github.com/3975481
>
> Please give it a shot and let us know what breaks.
--------------------------------------------
http://about.me/kkrugler
+1 530-210-6378
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB