Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Flume and HDFS integration


+
Emile Kao 2012-11-29, 17:14
+
Brock Noland 2012-11-29, 17:18
+
Roman Shaposhnik 2012-11-30, 01:17
Copy link to this message
-
Re: Flume and HDFS integration
HI,

On Thu, Nov 29, 2012 at 7:17 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> On Thu, Nov 29, 2012 at 9:18 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
>> 1) It's a sequence file, you can change it a text file if you want. See
>> FileType here http://flume.apache.org/FlumeUserGuide.html#hdfs-sink
>
> Don't you also have to change a serialization format to get rid of the binary
> structure completely? IOW, you'd have to add something like:
>     agent.sinks.hdfsSink.hdfs.serializer > org.apache.flume.serialization.BodyTextEventSerializer

BodyTextEventSerializer is the default serializer. Serializers decide
how to turn Events into records while fileType decides what type of
file the event is written to.

Brock
+
Emile Kao 2012-11-30, 08:51
+
Roman Shaposhnik 2012-12-01, 00:40
+
Brock Noland 2012-11-30, 12:40
+
Emile Kao 2012-12-03, 09:51
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB