Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - Straggler problem in Accumulo BatchScans


Copy link to this message
-
Straggler problem in Accumulo BatchScans
Slater, David M. 2013-08-21, 23:09
Hey, I have a 7 node network running accumulo 1.4.1 and hadoop 1.0.4.

When I run large BatchScanner operations, the number of tablets scanned per node is not uniform, leading to the overloaded nodes taking much longer to finish than the others. For queries that require all of the scans to finish before returning, this is a major latency issue. What are some practical means of load-balancing this to reduce delay?

Is it possible for tablets to be hosted on multiple tablet servers, up to the replication factor of the underlying hdfs? Are there reasons this might be an undesirable design?

Thanks in advance,
David