Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Writing AvroSequenceFiles to HDFS


Copy link to this message
-
Writing AvroSequenceFiles to HDFS
Hi,

I'm wondering if there's an easy (or at least recommended) way to write
AvroSequenceFile's to HDFS using Flume.

By creating a new serializer I'm able to write Avro objects to a sequence
file, but they're not wrapped up in AvroKey or AvroValue and the schema
isn't in the sequence file header.

I can't explicitly output AvroKey/Values from my serializer as they're not
Writables.

Before I plunge into HDFS sink code, is there a better way?

Many thanks,

Tom

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB