Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Tabletserver message "Running low on memory"


Copy link to this message
-
Tabletserver message "Running low on memory"
On an Accumulo 1.4.2 I've gotten "[tabletserver.TabletServer] WARN: Running
low on memory" 5 times in the last two days on just one of my 6 datanodes.
That datanode is hosting ~30% of the data, as 2 datanodes had dropped from
the cluster due to a network issue some time ago and hasn't entirely
rebalanced.  Current volume is only 140 million.

Ingest rates has been pretty constant at a light 200 per second.

Not knowing how Accumulo uses its java heap space, I opted to start with a
stock memory config and used the 3GB example config files, which I see
allocates only 1GB to the TabletServer. The server has 24GB RAM and
currently is using only 10GB total between Accumulo and HDFS, so there's
plenty of free memory to spare.

Is ACCUMULO_TSERVER_OPTS in accumulo-env.sh the tunable I should target to
alleviate these warnings?

Unfortunately being ops-configured nodes, no JDKs are installed nor is it a
possibility to do so in order to monitor the JVM itself for better
information.

Thanks in advance,
Terry
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB