Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Writing AvroSequenceFiles to HDFS

Copy link to this message
Writing AvroSequenceFiles to HDFS
Tom Davis 2014-03-03, 07:58

I'm wondering if there's an easy (or at least recommended) way to write
AvroSequenceFile's to HDFS using Flume.

By creating a new serializer I'm able to write Avro objects to a sequence
file, but they're not wrapped up in AvroKey or AvroValue and the schema
isn't in the sequence file header.

I can't explicitly output AvroKey/Values from my serializer as they're not

Before I plunge into HDFS sink code, is there a better way?

Many thanks,