Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Writing AvroSequenceFiles to HDFS


Copy link to this message
-
Writing AvroSequenceFiles to HDFS
Hi,

I'm wondering if there's an easy (or at least recommended) way to write
AvroSequenceFile's to HDFS using Flume.

By creating a new serializer I'm able to write Avro objects to a sequence
file, but they're not wrapped up in AvroKey or AvroValue and the schema
isn't in the sequence file header.

I can't explicitly output AvroKey/Values from my serializer as they're not
Writables.

Before I plunge into HDFS sink code, is there a better way?

Many thanks,

Tom