Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> Max tablet size & Pre-splitting


Copy link to this message
-
Max tablet size & Pre-splitting
First, a quick question: For Accumulo 1.4.2, is there a maximum size that tablet can have? In other words, if I was to do something like table.split.threshold=1000G, would that actually allow the tablet to grow to that size, or is there some static maximum, like 2G that a tablet can have?

The reason I ask this is that I’m doing time-based presplitting of tables, so that I add a set of split points when I get to a new time range (or one of the tablets reach a certain size), and then transfer all of my ingest to the new set of tablets created. This keeps me from needing to do any table splits involving data. Therefore, I would like to set the table split threshold arbitrarily high, so that my presplitting algorithm can do all the work.

Second, is there a preferred way to estimate the tablet sizes from the Java API? I have the Ingestion application using my split points and mutation.numBytes() to keep track of the number of bytes per tablet. Should I be using mutation.memory() instead? Or is there a more direct way via connector.tableOperations() or some other mechanism to determine the size of the tablet?

Thanks,
David
+
John Vines 2013-10-28, 18:11
+
Slater, David M. 2013-10-28, 18:50
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB