Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> xceiver count, regionserver shutdown


Copy link to this message
-
xceiver count, regionserver shutdown
I am trying to resolve an issue with my cluster when I am loading a bunch of data into HBase. I am reaching the "xciever" limit on the data nodes. Currently I have this set to 4096. The data node is logging "xceiverCount 4097 exceeds the limit of concurrent xcievers 4096". The regionservers eventually shut down. I have read the various threads on this issue.

I have 4 datanodes/regionservers. Each regionserver has only around 200 regions. The table has 2 column families. I have the region file size set to 500mb, and I'm using Snappy compression. This problem is occurring on HBase 0.90.4 and Hadoop 0.20.2 (both Cloudera cdh3u3).

From what I have read, the number of regions on a node can cause the xceiver limit to be reached, but it doesn't seem like I have an excessive number of regions. I want the table to scale higher, so simply upping the xceiver limit could perhaps get my table functional for now, but it seems it will only be a temporary fix.

Are number of regions the only factor that can cause this problem, or are there other factors involved that I may be able to adjust?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB