Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> flume 1.4.0 avro source/sink with hdfs sink configuration - no hdfs files created


+
Suhas Satish 2013-09-12, 02:55
+
Suhas Satish 2013-09-12, 03:08
+
Suhas Satish 2013-09-12, 03:16
+
Babu, Prashanth 2013-09-12, 04:15
+
Suhas Satish 2013-09-12, 17:34
+
Paul Chavez 2013-09-12, 17:48
+
Suhas Satish 2013-09-12, 17:57
Copy link to this message
-
RE: flume 1.4.0 avro source/sink with hdfs sink configuration - no hdfs files created
I think you can do this on one node, but you will need to run two instances of flume, each with a different agent name.

Paul

From: Suhas Satish [mailto:[EMAIL PROTECTED]]
Sent: Thursday, September 12, 2013 10:58 AM
To: [EMAIL PROTECTED]
Subject: Re: flume 1.4.0 avro source/sink with hdfs sink configuration - no hdfs files created

Thanks. Yes I was trying to set it up on a single node. If it cannot be done, I can go to 2 different nodes, but that would add additional complexities which I'd like to avoid if possible.

My original intent was to test if the Avro Source/Avro Sink interface can work with SSL enabled (hence the extra hop) and if it can, can it use the ssl_keystore and ssl_truststore already available from a secure hadoop cluster.

On Thu, Sep 12, 2013 at 10:48 AM, Paul Chavez <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
et this all up on a single node? If so, why are you adding in an extra Avro hop?

In practice this setup should be on two nodes, one acting as the 'agent' with the exec sour

Cheers,
Suhas.
+
Babu, Prashanth 2013-09-13, 02:16
+
Mahadevappa, Shobha 2013-09-13, 07:24
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB