Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> header record for each created hdfs file


+
Raymond Ng 2012-09-24, 16:05
Copy link to this message
-
Re: header record for each created hdfs file
Hi Ray,
Have you taken a look @ the EventSerializer interface? It lets you write a
plugin to customize a header or trailer on a per-file basis, in addition to
arbitrary serialization of the Flume Event objects.

http://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/serialization/EventSerializer.html
http://flume.apache.org/FlumeUserGuide.html#hdfs-sink

Regards,
Mike
On Mon, Sep 24, 2012 at 9:05 AM, Raymond Ng <[EMAIL PROTECTED]> wrote:

> Hi
>
> is it possible to include a header record for each hdfs file that the hdfs
> sink rolls over?
>
> I have a large csv file to put into hdfs but I'd like to have the csv
> columns written to the top of each of the rolled over file
>
> thanks
>
> --
> Rgds
> Ray
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB