Flume, mail # user - Writing AvroSequenceFiles to HDFS - 2014-03-03, 07:58
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Plain View
Copy link to this message
-
Writing AvroSequenceFiles to HDFS
Hi,

I'm wondering if there's an easy (or at least recommended) way to write
AvroSequenceFile's to HDFS using Flume.

By creating a new serializer I'm able to write Avro objects to a sequence
file, but they're not wrapped up in AvroKey or AvroValue and the schema
isn't in the sequence file header.

I can't explicitly output AvroKey/Values from my serializer as they're not
Writables.

Before I plunge into HDFS sink code, is there a better way?

Many thanks,

Tom

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB