Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Re: Number of concurrent writer to HDFS


Copy link to this message
-
Re: Number of concurrent writer to HDFS
Yanbo Liang 2012-08-06, 06:14
You can use scribe or flume to collect log data and integrated with hadoop.

2012/8/4 Nguyen Manh Tien <[EMAIL PROTECTED]>

> Hi,
> I plan to streaming logs data HDFS using many writer, each writer write a
> stream of data to a HDFS file (may rotate)
>
> I wonder how many concurrent writer i should use?
> And if you have that experience please share to me : hadoop cluster size,
> number of writer, replication.
>
> Thanks.
> Tien
>
+
Alex Baranau 2012-08-06, 14:58
+
Nguyen Manh Tien 2012-08-07, 01:35
+
Yanbo Liang 2012-08-07, 06:53
+
Nguyen Manh Tien 2012-08-08, 14:14
+
alo alt 2012-08-09, 06:48