Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Using Python and Flume to store avro data


Copy link to this message
-
Using Python and Flume to store avro data
Hi,

I've been spending quite a few hours trying to push avro data to Flume
so i can store it on HDFS, this all with Python.
It seems like something that is impossible for now, since the only  way
to push avro data to Flume is by the use of deprecated thrift binding
that look pretty cumbersome to get working.
I would like to know what's the best way to import avro data into Flume
with Python? Maybe Flume isnt the right tool and I should use something
else? My goal is to have multiple python workers pushing data to HDFS
which ( by means of Flume in this case ) consolidates this all in 1 file
there.

Any thoughts?

Thanks!

Bart
+
Hari Shreedharan 2012-11-08, 18:51
+
Bart Verwilst 2012-11-08, 18:57
+
Hari Shreedharan 2012-11-08, 19:06
+
Bart Verwilst 2012-11-08, 21:02
+
Hari Shreedharan 2012-11-08, 21:12
+
Bart Verwilst 2012-11-08, 21:34
+
Hari Shreedharan 2012-11-08, 21:50
+
Bart Verwilst 2012-11-08, 22:49
+
Brock Noland 2012-11-09, 01:30
+
Juhani Connolly 2012-11-09, 01:46
+
Camp, Roy 2012-11-12, 19:52
+
Andrew Jones 2012-11-13, 09:28
+
Bart Verwilst 2012-11-16, 10:54