Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> new log.dirs property (as opposed to log.dir)


+
Jason Rosenberg 2013-06-20, 05:25
+
Jun Rao 2013-06-20, 05:48
+
Jason Rosenberg 2013-06-20, 06:02
+
Jun Rao 2013-06-20, 14:53
Copy link to this message
-
Re: new log.dirs property (as opposed to log.dir)
The intention is to allow the use of multiple disks without RAID or
logical volume management. We have found that there are a lot of
downsides to RAID--in particular a huge throughput hit. Since we
already have a parallelism model due to partitioning and a fault
tolerance model with replication RAID doesn't actually buy much. With
this feature you can directly mount multiple disks as their own
directory and the server will randomly assign partitions to them.

Obviously this will only work well if there are enough high-throughput
partitions to make load balance evenly (e.g. if you have only one big
partition per server then this isn't going to work).

-Jay

On Wed, Jun 19, 2013 at 11:01 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

 
+
Jason Rosenberg 2013-06-20, 19:59
+
Jay Kreps 2013-06-20, 20:21
+
Jason Rosenberg 2013-06-20, 22:15
+
Jason Rosenberg 2013-08-14, 20:50
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB