Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Number of concurrent writer to HDFS


Copy link to this message
-
Re: Number of concurrent writer to HDFS
You can use scribe or flume to collect log data and integrated with hadoop.

2012/8/4 Nguyen Manh Tien <[EMAIL PROTECTED]>

> Hi,
> I plan to streaming logs data HDFS using many writer, each writer write a
> stream of data to a HDFS file (may rotate)
>
> I wonder how many concurrent writer i should use?
> And if you have that experience please share to me : hadoop cluster size,
> number of writer, replication.
>
> Thanks.
> Tien
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB