You can use scribe or flume to collect log data and integrated with hadoop.
2012/8/4 Nguyen Manh Tien <[EMAIL PROTECTED]>
> I plan to streaming logs data HDFS using many writer, each writer write a
> stream of data to a HDFS file (may rotate)
> I wonder how many concurrent writer i should use?
> And if you have that experience please share to me : hadoop cluster size,
> number of writer, replication.